You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Derek Chen-Becker <de...@chen-becker.org> on 2023/08/11 13:46:21 UTC

[Discuss] cleaning up build temp files

Hi,

While doing some local testing, I noticed that my /tmp drive completely
filled with test artifact files (e.g. data directories, logs, commit logs,
etc). Mick pointed out that we do attempt to do some "find" based cleanup
in CI (
https://github.com/apache/cassandra-builds/blob/trunk/jenkins-dsl/cassandra_job_dsl_seed.groovy#L437-439),
but I was wondering if it might be better to do the following for direct
ant builds:

1. If TMPDIR is set, use it. It does not appear to be honored, currently,
so I need to do some analysis of what would need to be done here
2. If TMPDIR is not set, use "mktemp" to create a temp directory and set
TMPDIR with that directory
3. Update the "ant clean" task to delete TMPDIR when we've generated it, or
attempt the find-based cleanup if TMPDIR was provided

Does anyone know if there are any hard-coded assumptions that test files
will live directly under /tmp?

Thanks,

Derek

-- 
+---------------------------------------------------------------+
| Derek Chen-Becker                                             |
| GPG Key available at https://keybase.io/dchenbecker and       |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---------------------------------------------------------------+

Re: [Discuss] cleaning up build temp files

Posted by Mick Semb Wever <mc...@apache.org>.
On Sun, 13 Aug 2023 at 16:48, Josh McKenzie <jm...@apache.org> wrote:

> Why not use "/${CASS_BUILD_TMP}/cassandra.<RAND>" on a given run and then
> on subsequent runs "rm -rf f/${CASS_BUILD_TMP}/cassandra.*"? If
> CASS_BUILD_TMP is not defined, default to /tmp.
>


I think we want/need relative paths, e.g. "build/tmp", and if the path is
in a mounted volume there can be another container still running.

No objections in theory, and this isn't difficult stuff, just a few
variations to deal with (that we don't have automated CI over).

Re: [Discuss] cleaning up build temp files

Posted by Josh McKenzie <jm...@apache.org>.
Why not use "/${CASS_BUILD_TMP}/cassandra.<RAND>" on a given run and then on subsequent runs "rm -rf f/${CASS_BUILD_TMP}/cassandra.*"? If CASS_BUILD_TMP is not defined, default to /tmp.

"ant clean" can also wipe it.

If it's a safe assumption that we only ever need 1 instance of data in that space (i.e. we won't have 2 builds / tests running in a single container concurrently) it seems the above would solve the problem. Different environments (circle, ASF, etc) could define CASS_BUILD_TMP differently if needed for their env and problem is solved.

On Sun, Aug 13, 2023, at 10:23 AM, Mick Semb Wever wrote:
> 
>> While doing some local testing, I noticed that my /tmp drive completely filled with test artifact files (e.g. data directories, logs, commit logs, etc). Mick pointed out that we do attempt to do some "find" based cleanup in CI (https://github.com/apache/cassandra-builds/blob/trunk/jenkins-dsl/cassandra_job_dsl_seed.groovy#L437-439), but I was wondering if it might be better to do the following for direct ant builds:
>> 
>> 1. If TMPDIR is set, use it. It does not appear to be honored, currently, so I need to do some analysis of what would need to be done here
>> 2. If TMPDIR is not set, use "mktemp" to create a temp directory and set TMPDIR with that directory
>> 3. Update the "ant clean" task to delete TMPDIR when we've generated it, or attempt the find-based cleanup if TMPDIR was provided
>> 
>> Does anyone know if there are any hard-coded assumptions that test files will live directly under /tmp?
> 
> 
> This will need testing with in-tree scripts, ci-cassandra, and circleci  :(
>  
> What comes to mind:
>  - TMPDIR works best today with the python and scripting stuff
>  - setting TMPDIR can break tests, hence unit test script set instead $TMP_DIR which is passed to `-Dtmp.dir=…`
>  - /tmp is often set up to be a more appropropriate fs (and volume size)
>  - it is hard to customise everything
>  - it needs to work locally on your machine as well as in docker containers, as well as CI
> 
> If we want something that is wiped by `ant clean` I would suggest using the build/tmp directory by default.
> In-tree scripts do this for unit tests: https://github.com/apache/cassandra/blob/trunk/.build/run-tests.sh#L160
>  but are not yet doing it for the dtests: https://github.com/apache/cassandra/blob/trunk/.build/run-python-dtests.sh#L58 
> 
> So I don't think we need (3). If the caller has specified TMPDIR it is then their responsibility to clean it.
> 
> We can probably avoid trying to set TMPDIR, instead defaulting the `tmp.dir` property to  the build/tmp directory.
> 
> The goal of any changes in build.xml should be, in addition to providing the best dev exp, to simplify the testing and CI layers above it.

Re: [Discuss] cleaning up build temp files

Posted by Brandon Williams <dr...@gmail.com>.
I will try to take a look tomorrow.

Kind Regards,
Brandon

On Sun, Aug 20, 2023 at 10:38 AM Derek Chen-Becker
<de...@chen-becker.org> wrote:
>
> Hi,
>
> Would someone be able to review the trunk patch for https://issues.apache.org/jira/browse/CASSANDRA-18750? I'm fighting CircleCI to try and get anything to pass (everything fails with "Concurrency limit"), but builds and tests work locally (via "ant test" and the docker "run-tests.sh").
>
> Cheers,
>
> Derek
>
> On Tue, Aug 15, 2023 at 8:13 AM Derek Chen-Becker <de...@chen-becker.org> wrote:
>>
>> Thanks, I thought I had updated against trunk but apparently missed it. It's working now.
>>
>> On Tue, Aug 15, 2023 at 7:27 AM Brandon Williams <dr...@gmail.com> wrote:
>>>
>>> On Tue, Aug 15, 2023 at 8:13 AM Derek Chen-Becker <de...@chen-becker.org> wrote:
>>> >
>>> > Also, I noticed that the instructions on https://cassandra.apache.org/_/development/how_to_commit.html mention "ant jar check" as a recommended step, but there's not "check" target. Does it mean "checkstyle"?
>>>
>>> It exists in the cassandra-5.0 and trunk branches after CASSANDRA-18618
>>
>>
>>
>> --
>> +---------------------------------------------------------------+
>> | Derek Chen-Becker                                             |
>> | GPG Key available at https://keybase.io/dchenbecker and       |
>> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
>> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
>> +---------------------------------------------------------------+
>>
>
>
> --
> +---------------------------------------------------------------+
> | Derek Chen-Becker                                             |
> | GPG Key available at https://keybase.io/dchenbecker and       |
> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
> +---------------------------------------------------------------+
>

Re: [Discuss] cleaning up build temp files

Posted by Derek Chen-Becker <de...@chen-becker.org>.
Hi,

Would someone be able to review the trunk patch for
https://issues.apache.org/jira/browse/CASSANDRA-18750? I'm fighting
CircleCI to try and get anything to pass (everything fails with
"Concurrency limit"), but builds and tests work locally (via "ant test" and
the docker "run-tests.sh").

Cheers,

Derek

On Tue, Aug 15, 2023 at 8:13 AM Derek Chen-Becker <de...@chen-becker.org>
wrote:

> Thanks, I thought I had updated against trunk but apparently missed it.
> It's working now.
>
> On Tue, Aug 15, 2023 at 7:27 AM Brandon Williams <dr...@gmail.com> wrote:
>
>> On Tue, Aug 15, 2023 at 8:13 AM Derek Chen-Becker <de...@chen-becker.org>
>> wrote:
>> >
>> > Also, I noticed that the instructions on
>> https://cassandra.apache.org/_/development/how_to_commit.html mention
>> "ant jar check" as a recommended step, but there's not "check" target. Does
>> it mean "checkstyle"?
>>
>> It exists in the cassandra-5.0 and trunk branches after CASSANDRA-18618
>>
>
>
> --
> +---------------------------------------------------------------+
> | Derek Chen-Becker                                             |
> | GPG Key available at https://keybase.io/dchenbecker and       |
> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
> +---------------------------------------------------------------+
>
>

-- 
+---------------------------------------------------------------+
| Derek Chen-Becker                                             |
| GPG Key available at https://keybase.io/dchenbecker and       |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---------------------------------------------------------------+

Re: [Discuss] cleaning up build temp files

Posted by Derek Chen-Becker <de...@chen-becker.org>.
Thanks, I thought I had updated against trunk but apparently missed it.
It's working now.

On Tue, Aug 15, 2023 at 7:27 AM Brandon Williams <dr...@gmail.com> wrote:

> On Tue, Aug 15, 2023 at 8:13 AM Derek Chen-Becker <de...@chen-becker.org>
> wrote:
> >
> > Also, I noticed that the instructions on
> https://cassandra.apache.org/_/development/how_to_commit.html mention
> "ant jar check" as a recommended step, but there's not "check" target. Does
> it mean "checkstyle"?
>
> It exists in the cassandra-5.0 and trunk branches after CASSANDRA-18618
>


-- 
+---------------------------------------------------------------+
| Derek Chen-Becker                                             |
| GPG Key available at https://keybase.io/dchenbecker and       |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---------------------------------------------------------------+

Re: [Discuss] cleaning up build temp files

Posted by Brandon Williams <dr...@gmail.com>.
On Tue, Aug 15, 2023 at 8:13 AM Derek Chen-Becker <de...@chen-becker.org> wrote:
>
> Also, I noticed that the instructions on https://cassandra.apache.org/_/development/how_to_commit.html mention "ant jar check" as a recommended step, but there's not "check" target. Does it mean "checkstyle"?

It exists in the cassandra-5.0 and trunk branches after CASSANDRA-18618

Re: [Discuss] cleaning up build temp files

Posted by Derek Chen-Becker <de...@chen-becker.org>.
Thanks Brandon, that did make things simple. PR against trunk is here for
review, and once we get it ship-shape I can create patches against the
other version branches:

https://github.com/apache/cassandra/pull/2593

Also, I noticed that the instructions on
https://cassandra.apache.org/_/development/how_to_commit.html mention "ant
jar check" as a recommended step, but there's not "check" target. Does it
mean "checkstyle"?

Thanks,

Derek

On Tue, Aug 15, 2023 at 4:34 AM Brandon Williams <dr...@gmail.com> wrote:

> I think most of those use InMemoryAuditLogger so it's not a problem,
> but we can call DatabaseDescriptor.setAuditLoggingOptions to change
> the dir, the same way the AuditLoggerTest.testJMXArchive method does.
>
> Kind Regards,
> Brandon
>
> On Mon, Aug 14, 2023 at 11:23 PM Derek Chen-Becker
> <de...@chen-becker.org> wrote:
> >
> > I think I'm 98% done with the needed changes (or at least a first pass;
> I'm sure I'll get feedback), but I've hit an issue with AuditLoggerTest.
> While AuditLogOptions has a String property for "audit_logs_dir", the test
> calls StorageService.enableAuditLogs throught the test methods, and that
> method does not take a parameter for the log directory. As a result, the
> AuditLogOptions instance that is created always uses the default audit log
> directory, which just happens to be the "audit" subdirectory of the project
> root. This is getting into territory I'm not that familiar with, but it
> seems like there might be three ways to fix this:
> >
> > 1. Write a new overload of StorageService.enableAuditLog that takes an
> AuditLogOptions and delegate the other overloads to it
> > 2. Add a new overload that also takes a Path or File pointing to the
> audit log directory
> > 3. Set the "cassandra.logdir.audit" property in the ant build, similar
> to "tmp.dir"/"java.io.tmpdir"
> >
> > Thoughts?
> >
> > Derek
> >
> > On Sun, Aug 13, 2023 at 3:12 PM Josh McKenzie <jm...@apache.org>
> wrote:
> >>
> >> There's also tests that hardcode
> >>
> >> I started mentally twitching when I hit that point in the sentence.
> >>
> >> Kill them with fire.
> >>
> >> On Sun, Aug 13, 2023, at 4:51 PM, Mick Semb Wever wrote:
> >>
> >>
> >>
> https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L717-L719
> >>
> https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L757-L759
> >>
> >> Can I open a ticket to track fixes for these and any other issues I run
> into while moving to using "build/tmp"?
> >>
> >>
> >>
> >> Go for it. :-)
> >> There's also tests that hardcode other paths that breaks the use of
> `build.dir`
> >>
> >>
> >
> >
> > --
> > +---------------------------------------------------------------+
> > | Derek Chen-Becker                                             |
> > | GPG Key available at https://keybase.io/dchenbecker and       |
> > | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
> > | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
> > +---------------------------------------------------------------+
> >
>


-- 
+---------------------------------------------------------------+
| Derek Chen-Becker                                             |
| GPG Key available at https://keybase.io/dchenbecker and       |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---------------------------------------------------------------+

Re: [Discuss] cleaning up build temp files

Posted by Brandon Williams <dr...@gmail.com>.
I think most of those use InMemoryAuditLogger so it's not a problem,
but we can call DatabaseDescriptor.setAuditLoggingOptions to change
the dir, the same way the AuditLoggerTest.testJMXArchive method does.

Kind Regards,
Brandon

On Mon, Aug 14, 2023 at 11:23 PM Derek Chen-Becker
<de...@chen-becker.org> wrote:
>
> I think I'm 98% done with the needed changes (or at least a first pass; I'm sure I'll get feedback), but I've hit an issue with AuditLoggerTest. While AuditLogOptions has a String property for "audit_logs_dir", the test calls StorageService.enableAuditLogs throught the test methods, and that method does not take a parameter for the log directory. As a result, the AuditLogOptions instance that is created always uses the default audit log directory, which just happens to be the "audit" subdirectory of the project root. This is getting into territory I'm not that familiar with, but it seems like there might be three ways to fix this:
>
> 1. Write a new overload of StorageService.enableAuditLog that takes an AuditLogOptions and delegate the other overloads to it
> 2. Add a new overload that also takes a Path or File pointing to the audit log directory
> 3. Set the "cassandra.logdir.audit" property in the ant build, similar to "tmp.dir"/"java.io.tmpdir"
>
> Thoughts?
>
> Derek
>
> On Sun, Aug 13, 2023 at 3:12 PM Josh McKenzie <jm...@apache.org> wrote:
>>
>> There's also tests that hardcode
>>
>> I started mentally twitching when I hit that point in the sentence.
>>
>> Kill them with fire.
>>
>> On Sun, Aug 13, 2023, at 4:51 PM, Mick Semb Wever wrote:
>>
>>
>> https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L717-L719
>> https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L757-L759
>>
>> Can I open a ticket to track fixes for these and any other issues I run into while moving to using "build/tmp"?
>>
>>
>>
>> Go for it. :-)
>> There's also tests that hardcode other paths that breaks the use of `build.dir`
>>
>>
>
>
> --
> +---------------------------------------------------------------+
> | Derek Chen-Becker                                             |
> | GPG Key available at https://keybase.io/dchenbecker and       |
> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
> +---------------------------------------------------------------+
>

Re: [Discuss] cleaning up build temp files

Posted by Derek Chen-Becker <de...@chen-becker.org>.
I think I'm 98% done with the needed changes (or at least a first pass; I'm
sure I'll get feedback), but I've hit an issue with AuditLoggerTest. While
AuditLogOptions has a String property for "audit_logs_dir", the test calls
StorageService.enableAuditLogs throught the test methods, and that method
does not take a parameter for the log directory. As a result, the
AuditLogOptions instance that is created always uses the default audit log
directory, which just happens to be the "audit" subdirectory of the project
root. This is getting into territory I'm not that familiar with, but it
seems like there might be three ways to fix this:

1. Write a new overload of StorageService.enableAuditLog that takes an
AuditLogOptions and delegate the other overloads to it
2. Add a new overload that also takes a Path or File pointing to the audit
log directory
3. Set the "cassandra.logdir.audit" property in the ant build, similar to
"tmp.dir"/"java.io.tmpdir"

Thoughts?

Derek

On Sun, Aug 13, 2023 at 3:12 PM Josh McKenzie <jm...@apache.org> wrote:

> There's also tests that hardcode
>
> I started mentally twitching when I hit that point in the sentence.
>
> *Kill them with fire.*
>
> On Sun, Aug 13, 2023, at 4:51 PM, Mick Semb Wever wrote:
>
>
>
> https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L717-L719
>
> https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L757-L759
>
> Can I open a ticket to track fixes for these and any other issues I run
> into while moving to using "build/tmp"?
>
>
>
> Go for it. :-)
> There's also tests that hardcode other paths that breaks the use of
> `build.dir`
>
>
>

-- 
+---------------------------------------------------------------+
| Derek Chen-Becker                                             |
| GPG Key available at https://keybase.io/dchenbecker and       |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---------------------------------------------------------------+

Re: [Discuss] cleaning up build temp files

Posted by Josh McKenzie <jm...@apache.org>.
> There's also tests that hardcode
I started mentally twitching when I hit that point in the sentence.

**Kill them with fire.**

On Sun, Aug 13, 2023, at 4:51 PM, Mick Semb Wever wrote:
>> 
>> https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L717-L719
>> https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L757-L759
>> 
>> Can I open a ticket to track fixes for these and any other issues I run into while moving to using "build/tmp"?
> 
> 
> Go for it. :-) 
> There's also tests that hardcode other paths that breaks the use of `build.dir`

Re: [Discuss] cleaning up build temp files

Posted by Mick Semb Wever <mc...@apache.org>.
>
>
>
> https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L717-L719
>
> https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L757-L759
>
> Can I open a ticket to track fixes for these and any other issues I run
> into while moving to using "build/tmp"?
>


Go for it. :-)
There's also tests that hardcode other paths that breaks the use of
`build.dir`

Re: [Discuss] cleaning up build temp files

Posted by Derek Chen-Becker <de...@chen-becker.org>.
OK, I already found some places where "/tmp" is hard-coded:

https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L717-L719
https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L757-L759

Can I open a ticket to track fixes for these and any other issues I run
into while moving to using "build/tmp"?

Thanks,

Derek

On Sun, Aug 13, 2023 at 10:13 AM Josh McKenzie <jm...@apache.org> wrote:

> I think we want/need relative paths, e.g. "build/tmp", and if the path is
> in a mounted volume there can be another container still running.
>
> Sure. The specifics of *what* path isn't interesting to me.
>
> The pattern of:
> 1. Let env declare where TEMP lives
> 2. Write things to TEMP
> 3. Delete things from TEMP every time we run a new suite or do "ant clean"
>
> Is.
>
> Could also take it a step further and let env declare RESULTS_PATH for
> things they want to be durable and add an "ant clean-results" target.
>
> On Sun, Aug 13, 2023, at 11:33 AM, Derek Chen-Becker wrote:
>
> Nevermind, I found "tmp.dir"
>
> On Sun, Aug 13, 2023 at 9:29 AM Derek Chen-Becker <de...@chen-becker.org>
> wrote:
>
> Cool,
>
> I'm a little confused. Is "tmp.dir" a custom Java property that we expose?
> I thought that the standard "property was "java.io.tmpdir". Let me take a
> stab at setting tmp.dir to build/tmp and see if I run into any issues (or
> still see any files in /tmp).
>
> Cheers,
>
> Derek
>
> On Sun, Aug 13, 2023 at 8:24 AM Mick Semb Wever <mc...@apache.org> wrote:
>
>
> While doing some local testing, I noticed that my /tmp drive completely
> filled with test artifact files (e.g. data directories, logs, commit logs,
> etc). Mick pointed out that we do attempt to do some "find" based cleanup
> in CI (
> https://github.com/apache/cassandra-builds/blob/trunk/jenkins-dsl/cassandra_job_dsl_seed.groovy#L437-439),
> but I was wondering if it might be better to do the following for direct
> ant builds:
>
> 1. If TMPDIR is set, use it. It does not appear to be honored, currently,
> so I need to do some analysis of what would need to be done here
> 2. If TMPDIR is not set, use "mktemp" to create a temp directory and set
> TMPDIR with that directory
> 3. Update the "ant clean" task to delete TMPDIR when we've generated it,
> or attempt the find-based cleanup if TMPDIR was provided
>
> Does anyone know if there are any hard-coded assumptions that test files
> will live directly under /tmp?
>
>
>
> This will need testing with in-tree scripts, ci-cassandra, and circleci  :(
>
> What comes to mind:
>  - TMPDIR works best today with the python and scripting stuff
>  - setting TMPDIR can break tests, hence unit test script set instead
> $TMP_DIR which is passed to `-Dtmp.dir=…`
>  - /tmp is often set up to be a more appropropriate fs (and volume size)
>  - it is hard to customise everything
>  - it needs to work locally on your machine as well as in docker
> containers, as well as CI
>
> If we want something that is wiped by `ant clean` I would suggest using
> the build/tmp directory by default.
> In-tree scripts do this for unit tests:
> https://github.com/apache/cassandra/blob/trunk/.build/run-tests.sh#L160
>  but are not yet doing it for the dtests:
> https://github.com/apache/cassandra/blob/trunk/.build/run-python-dtests.sh#L58
>
>
> So I don't think we need (3). If the caller has specified TMPDIR it is
> then their responsibility to clean it.
>
> We can probably avoid trying to set TMPDIR, instead defaulting the
> `tmp.dir` property to  the build/tmp directory.
>
> The goal of any changes in build.xml should be, in addition to providing
> the best dev exp, to simplify the testing and CI layers above it.
>
>
>
> --
> +---------------------------------------------------------------+
> | Derek Chen-Becker                                             |
> | GPG Key available at https://keybase.io/dchenbecker and       |
> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
> +---------------------------------------------------------------+
>
>
>
> --
> +---------------------------------------------------------------+
> | Derek Chen-Becker                                             |
> | GPG Key available at https://keybase.io/dchenbecker and       |
> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
> +---------------------------------------------------------------+
>
>
>

-- 
+---------------------------------------------------------------+
| Derek Chen-Becker                                             |
| GPG Key available at https://keybase.io/dchenbecker and       |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---------------------------------------------------------------+

Re: [Discuss] cleaning up build temp files

Posted by Josh McKenzie <jm...@apache.org>.
> I think we want/need relative paths, e.g. "build/tmp", and if the path is in a mounted volume there can be another container still running.
Sure. The specifics of *what* path isn't interesting to me.

The pattern of:
1. Let env declare where TEMP lives
2. Write things to TEMP
3. Delete things from TEMP every time we run a new suite or do "ant clean"

Is.

Could also take it a step further and let env declare RESULTS_PATH for things they want to be durable and add an "ant clean-results" target.

On Sun, Aug 13, 2023, at 11:33 AM, Derek Chen-Becker wrote:
> Nevermind, I found "tmp.dir"
> 
> On Sun, Aug 13, 2023 at 9:29 AM Derek Chen-Becker <de...@chen-becker.org> wrote:
>> Cool,
>> 
>> I'm a little confused. Is "tmp.dir" a custom Java property that we expose? I thought that the standard "property was "java.io.tmpdir". Let me take a stab at setting tmp.dir to build/tmp and see if I run into any issues (or still see any files in /tmp).
>> 
>> Cheers,
>> 
>> Derek
>> 
>> On Sun, Aug 13, 2023 at 8:24 AM Mick Semb Wever <mc...@apache.org> wrote:
>>> 
>>>> While doing some local testing, I noticed that my /tmp drive completely filled with test artifact files (e.g. data directories, logs, commit logs, etc). Mick pointed out that we do attempt to do some "find" based cleanup in CI (https://github.com/apache/cassandra-builds/blob/trunk/jenkins-dsl/cassandra_job_dsl_seed.groovy#L437-439), but I was wondering if it might be better to do the following for direct ant builds:
>>>> 
>>>> 1. If TMPDIR is set, use it. It does not appear to be honored, currently, so I need to do some analysis of what would need to be done here
>>>> 2. If TMPDIR is not set, use "mktemp" to create a temp directory and set TMPDIR with that directory
>>>> 3. Update the "ant clean" task to delete TMPDIR when we've generated it, or attempt the find-based cleanup if TMPDIR was provided
>>>> 
>>>> Does anyone know if there are any hard-coded assumptions that test files will live directly under /tmp?
>>> 
>>> 
>>> This will need testing with in-tree scripts, ci-cassandra, and circleci  :(
>>>  
>>> What comes to mind:
>>>  - TMPDIR works best today with the python and scripting stuff
>>>  - setting TMPDIR can break tests, hence unit test script set instead $TMP_DIR which is passed to `-Dtmp.dir=…`
>>>  - /tmp is often set up to be a more appropropriate fs (and volume size)
>>>  - it is hard to customise everything
>>>  - it needs to work locally on your machine as well as in docker containers, as well as CI
>>> 
>>> If we want something that is wiped by `ant clean` I would suggest using the build/tmp directory by default.
>>> In-tree scripts do this for unit tests: https://github.com/apache/cassandra/blob/trunk/.build/run-tests.sh#L160
>>>  but are not yet doing it for the dtests: https://github.com/apache/cassandra/blob/trunk/.build/run-python-dtests.sh#L58 
>>> 
>>> So I don't think we need (3). If the caller has specified TMPDIR it is then their responsibility to clean it.
>>> 
>>> We can probably avoid trying to set TMPDIR, instead defaulting the `tmp.dir` property to  the build/tmp directory.
>>> 
>>> The goal of any changes in build.xml should be, in addition to providing the best dev exp, to simplify the testing and CI layers above it.
>>> 
>> 
>> 
>> --
>> +---------------------------------------------------------------+
>> | Derek Chen-Becker                                             |
>> | GPG Key available at https://keybase.io/dchenbecker and       |
>> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
>> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
>> +---------------------------------------------------------------+
>> 
> 
> 
> --
> +---------------------------------------------------------------+
> | Derek Chen-Becker                                             |
> | GPG Key available at https://keybase.io/dchenbecker and       |
> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
> +---------------------------------------------------------------+
> 

Re: [Discuss] cleaning up build temp files

Posted by Derek Chen-Becker <de...@chen-becker.org>.
Nevermind, I found "tmp.dir"

On Sun, Aug 13, 2023 at 9:29 AM Derek Chen-Becker <de...@chen-becker.org>
wrote:

> Cool,
>
> I'm a little confused. Is "tmp.dir" a custom Java property that we expose?
> I thought that the standard "property was "java.io.tmpdir". Let me take a
> stab at setting tmp.dir to build/tmp and see if I run into any issues (or
> still see any files in /tmp).
>
> Cheers,
>
> Derek
>
> On Sun, Aug 13, 2023 at 8:24 AM Mick Semb Wever <mc...@apache.org> wrote:
>
>>
>> While doing some local testing, I noticed that my /tmp drive completely
>>> filled with test artifact files (e.g. data directories, logs, commit logs,
>>> etc). Mick pointed out that we do attempt to do some "find" based cleanup
>>> in CI (
>>> https://github.com/apache/cassandra-builds/blob/trunk/jenkins-dsl/cassandra_job_dsl_seed.groovy#L437-439),
>>> but I was wondering if it might be better to do the following for direct
>>> ant builds:
>>>
>>> 1. If TMPDIR is set, use it. It does not appear to be honored,
>>> currently, so I need to do some analysis of what would need to be done here
>>> 2. If TMPDIR is not set, use "mktemp" to create a temp directory and set
>>> TMPDIR with that directory
>>> 3. Update the "ant clean" task to delete TMPDIR when we've generated it,
>>> or attempt the find-based cleanup if TMPDIR was provided
>>>
>>> Does anyone know if there are any hard-coded assumptions that test files
>>> will live directly under /tmp?
>>>
>>
>>
>> This will need testing with in-tree scripts, ci-cassandra, and circleci
>>  :(
>>
>> What comes to mind:
>>  - TMPDIR works best today with the python and scripting stuff
>>  - setting TMPDIR can break tests, hence unit test script set instead
>> $TMP_DIR which is passed to `-Dtmp.dir=…`
>>  - /tmp is often set up to be a more appropropriate fs (and volume size)
>>  - it is hard to customise everything
>>  - it needs to work locally on your machine as well as in docker
>> containers, as well as CI
>>
>> If we want something that is wiped by `ant clean` I would suggest using
>> the build/tmp directory by default.
>> In-tree scripts do this for unit tests:
>> https://github.com/apache/cassandra/blob/trunk/.build/run-tests.sh#L160
>>  but are not yet doing it for the dtests:
>> https://github.com/apache/cassandra/blob/trunk/.build/run-python-dtests.sh#L58
>>
>>
>> So I don't think we need (3). If the caller has specified TMPDIR it is
>> then their responsibility to clean it.
>>
>> We can probably avoid trying to set TMPDIR, instead defaulting the
>> `tmp.dir` property to  the build/tmp directory.
>>
>> The goal of any changes in build.xml should be, in addition to providing
>> the best dev exp, to simplify the testing and CI layers above it.
>>
>>
>
> --
> +---------------------------------------------------------------+
> | Derek Chen-Becker                                             |
> | GPG Key available at https://keybase.io/dchenbecker and       |
> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
> +---------------------------------------------------------------+
>
>

-- 
+---------------------------------------------------------------+
| Derek Chen-Becker                                             |
| GPG Key available at https://keybase.io/dchenbecker and       |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---------------------------------------------------------------+

Re: [Discuss] cleaning up build temp files

Posted by Derek Chen-Becker <de...@chen-becker.org>.
Cool,

I'm a little confused. Is "tmp.dir" a custom Java property that we expose?
I thought that the standard "property was "java.io.tmpdir". Let me take a
stab at setting tmp.dir to build/tmp and see if I run into any issues (or
still see any files in /tmp).

Cheers,

Derek

On Sun, Aug 13, 2023 at 8:24 AM Mick Semb Wever <mc...@apache.org> wrote:

>
> While doing some local testing, I noticed that my /tmp drive completely
>> filled with test artifact files (e.g. data directories, logs, commit logs,
>> etc). Mick pointed out that we do attempt to do some "find" based cleanup
>> in CI (
>> https://github.com/apache/cassandra-builds/blob/trunk/jenkins-dsl/cassandra_job_dsl_seed.groovy#L437-439),
>> but I was wondering if it might be better to do the following for direct
>> ant builds:
>>
>> 1. If TMPDIR is set, use it. It does not appear to be honored, currently,
>> so I need to do some analysis of what would need to be done here
>> 2. If TMPDIR is not set, use "mktemp" to create a temp directory and set
>> TMPDIR with that directory
>> 3. Update the "ant clean" task to delete TMPDIR when we've generated it,
>> or attempt the find-based cleanup if TMPDIR was provided
>>
>> Does anyone know if there are any hard-coded assumptions that test files
>> will live directly under /tmp?
>>
>
>
> This will need testing with in-tree scripts, ci-cassandra, and circleci  :(
>
> What comes to mind:
>  - TMPDIR works best today with the python and scripting stuff
>  - setting TMPDIR can break tests, hence unit test script set instead
> $TMP_DIR which is passed to `-Dtmp.dir=…`
>  - /tmp is often set up to be a more appropropriate fs (and volume size)
>  - it is hard to customise everything
>  - it needs to work locally on your machine as well as in docker
> containers, as well as CI
>
> If we want something that is wiped by `ant clean` I would suggest using
> the build/tmp directory by default.
> In-tree scripts do this for unit tests:
> https://github.com/apache/cassandra/blob/trunk/.build/run-tests.sh#L160
>  but are not yet doing it for the dtests:
> https://github.com/apache/cassandra/blob/trunk/.build/run-python-dtests.sh#L58
>
>
> So I don't think we need (3). If the caller has specified TMPDIR it is
> then their responsibility to clean it.
>
> We can probably avoid trying to set TMPDIR, instead defaulting the
> `tmp.dir` property to  the build/tmp directory.
>
> The goal of any changes in build.xml should be, in addition to providing
> the best dev exp, to simplify the testing and CI layers above it.
>
>

-- 
+---------------------------------------------------------------+
| Derek Chen-Becker                                             |
| GPG Key available at https://keybase.io/dchenbecker and       |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---------------------------------------------------------------+

Re: [Discuss] cleaning up build temp files

Posted by Mick Semb Wever <mc...@apache.org>.
> While doing some local testing, I noticed that my /tmp drive completely
> filled with test artifact files (e.g. data directories, logs, commit logs,
> etc). Mick pointed out that we do attempt to do some "find" based cleanup
> in CI (
> https://github.com/apache/cassandra-builds/blob/trunk/jenkins-dsl/cassandra_job_dsl_seed.groovy#L437-439),
> but I was wondering if it might be better to do the following for direct
> ant builds:
>
> 1. If TMPDIR is set, use it. It does not appear to be honored, currently,
> so I need to do some analysis of what would need to be done here
> 2. If TMPDIR is not set, use "mktemp" to create a temp directory and set
> TMPDIR with that directory
> 3. Update the "ant clean" task to delete TMPDIR when we've generated it,
> or attempt the find-based cleanup if TMPDIR was provided
>
> Does anyone know if there are any hard-coded assumptions that test files
> will live directly under /tmp?
>


This will need testing with in-tree scripts, ci-cassandra, and circleci  :(

What comes to mind:
 - TMPDIR works best today with the python and scripting stuff
 - setting TMPDIR can break tests, hence unit test script set instead
$TMP_DIR which is passed to `-Dtmp.dir=…`
 - /tmp is often set up to be a more appropropriate fs (and volume size)
 - it is hard to customise everything
 - it needs to work locally on your machine as well as in docker
containers, as well as CI

If we want something that is wiped by `ant clean` I would suggest using the
build/tmp directory by default.
In-tree scripts do this for unit tests:
https://github.com/apache/cassandra/blob/trunk/.build/run-tests.sh#L160
 but are not yet doing it for the dtests:
https://github.com/apache/cassandra/blob/trunk/.build/run-python-dtests.sh#L58


So I don't think we need (3). If the caller has specified TMPDIR it is then
their responsibility to clean it.

We can probably avoid trying to set TMPDIR, instead defaulting the
`tmp.dir` property to  the build/tmp directory.

The goal of any changes in build.xml should be, in addition to providing
the best dev exp, to simplify the testing and CI layers above it.