You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Julian Foad <ju...@apache.org> on 2022/02/11 14:48:28 UTC
Help with FAIL: wc-queries-test 3: test query expectations
Re. pristines-on-demand. I have re-based pristines-on-demand on top of
multi-wc-format, resulting in a new branch 'pristines-on-demand-on-mwf'.
On 'pristines-on-demand-on-mwf':
> * Make pristines-on-demand behaviour conditional on WC format.
Mostly done in r1897977.
I could do with some help on a SQL test FAIL, when testing against the
older WC format, in wc-queries-test 3: test_query_expectations(). It
looks like this:
$ (SRC_DIR=$PWD; cd obj-dir/subversion/tests/libsvn_wc &&
./wc-queries-test --cleanup
--config-file=$SRC_DIR/subversion/tests/tests.conf
--srcdir=$SRC_DIR/subversion/tests/libsvn_wc --wc-format-version=1.14 3
| sed "s#$SRC_DIR/##")
subversion/tests/libsvn_wc/wc-queries-test.c:815: (apr_err=SVN_ERR_TEST_FAILED)
svn_tests: E200006: STMT_SELECT_UNREFERENCED_PRISTINES: Uses pristine
with only 0 index component: ((null))
SELECT checksum FROM pristine WHERE refcount = 0
subversion/tests/libsvn_wc/wc-queries-test.c:868: (apr_err=SVN_ERR_COMPOSED_ERROR)
svn_tests: E200042: Additional errors:
subversion/tests/libsvn_wc/wc-queries-test.c:868: (apr_err=SVN_ERR_SQLITE_CONSTRAINT)
svn_tests: E200035: |SCAN TABLE pristine
FAIL: wc-queries-test 3: test query expectations
The failure is within:
if (item->search
&& ((item->expression_vars < 2 && is_node_table(item->table))
|| (item->expression_vars < 1))
&& !is_result_table(item->table))
I have taken a look and cannot work out what it means. Maybe the
statement needs to be made to match the exception clause
"in_list(primary_key_statements, i)", but that's just a semi-educated guess.
Can anyone more familiar with SQL or that test help out here please?
- Julian
Re: Help with FAIL: wc-queries-test 3: test query expectations
Posted by Julian Foad <ju...@apache.org>.
Thank you very much, Daniel. Yes, that's spot on. Fixed in r1898002.
- Julian
Re: Help with FAIL: wc-queries-test 3: test query expectations
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Daniel Shahaf wrote on Fri, Feb 11, 2022 at 17:39:42 +0000:
> Julian Foad wrote on Fri, Feb 11, 2022 at 14:48:28 +0000:
> > On 'pristines-on-demand-on-mwf':
> ⋮
> > I could do with some help on a SQL test FAIL, when testing against the
> > older WC format, in wc-queries-test 3: test_query_expectations(). It
> > looks like this:
> >
> > $ (SRC_DIR=$PWD; cd obj-dir/subversion/tests/libsvn_wc &&
> > ./wc-queries-test --cleanup
> > --config-file=$SRC_DIR/subversion/tests/tests.conf
> > --srcdir=$SRC_DIR/subversion/tests/libsvn_wc --wc-format-version=1.14 3
> > | sed "s#$SRC_DIR/##")
> >
> > subversion/tests/libsvn_wc/wc-queries-test.c:815: (apr_err=SVN_ERR_TEST_FAILED)
> > svn_tests: E200006: STMT_SELECT_UNREFERENCED_PRISTINES: Uses pristine with only 0 index component: ((null))
> > SELECT checksum FROM pristine WHERE refcount = 0
> > subversion/tests/libsvn_wc/wc-queries-test.c:868: (apr_err=SVN_ERR_COMPOSED_ERROR)
> > svn_tests: E200042: Additional errors:
> > subversion/tests/libsvn_wc/wc-queries-test.c:868: (apr_err=SVN_ERR_SQLITE_CONSTRAINT)
> > svn_tests: E200035: |SCAN TABLE pristine
> > FAIL: wc-queries-test 3: test query expectations
> >
> > The failure is within:
> > if (item->search
> > && ((item->expression_vars < 2 && is_node_table(item->table))
> > || (item->expression_vars < 1))
> > && !is_result_table(item->table))
> >
> > I have taken a look and cannot work out what it means.
>
> In short, it's checking whether SQLite is using sufficiently many index
> columns. STMT_SELECT_UNREFERENCED_PRISTINES does a table scan without
> using an index, meaning that query does O(N) work where N is the number
> of rows in the table.
>
> To confirm that, does the test pass if you do «CREATE INDEX foo ON
> pristine(refcount)» when creating the SQLite database the test runs on?
>
> I haven't looked to see whether that's the right fix. That would depend
> on whether STMT_SELECT_UNREFERENCED_PRISTINES needs to be performant,
> and if it does, on what kind of index would best serve it without
> affecting other queries (e.g., INSERTs and DELETEs to that table).
>
> > Maybe the statement needs to be made to match the exception clause
> > "in_list(primary_key_statements, i)", but that's just a semi-educated guess.
>
> The statements in that list both have «LIMIT 1» on them. That'd be why
> they're exempted from the "all queries must use an index" check.
>
> >
>
> Excuse brevity.
Looked a bit more. The branch removes STMT_SELECT_UNREFERENCED_PRISTINES from slow_statements[]
unconditionally and then creates I_PRISTINE_UNREFERENCED only for f32
wc's. So, just add that statement to that array for f31 wc's.
Daniel
Re: Help with FAIL: wc-queries-test 3: test query expectations
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Julian Foad wrote on Fri, Feb 11, 2022 at 14:48:28 +0000:
> On 'pristines-on-demand-on-mwf':
⋮
> I could do with some help on a SQL test FAIL, when testing against the
> older WC format, in wc-queries-test 3: test_query_expectations(). It
> looks like this:
>
> $ (SRC_DIR=$PWD; cd obj-dir/subversion/tests/libsvn_wc &&
> ./wc-queries-test --cleanup
> --config-file=$SRC_DIR/subversion/tests/tests.conf
> --srcdir=$SRC_DIR/subversion/tests/libsvn_wc --wc-format-version=1.14 3
> | sed "s#$SRC_DIR/##")
>
> subversion/tests/libsvn_wc/wc-queries-test.c:815: (apr_err=SVN_ERR_TEST_FAILED)
> svn_tests: E200006: STMT_SELECT_UNREFERENCED_PRISTINES: Uses pristine with only 0 index component: ((null))
> SELECT checksum FROM pristine WHERE refcount = 0
> subversion/tests/libsvn_wc/wc-queries-test.c:868: (apr_err=SVN_ERR_COMPOSED_ERROR)
> svn_tests: E200042: Additional errors:
> subversion/tests/libsvn_wc/wc-queries-test.c:868: (apr_err=SVN_ERR_SQLITE_CONSTRAINT)
> svn_tests: E200035: |SCAN TABLE pristine
> FAIL: wc-queries-test 3: test query expectations
>
> The failure is within:
> if (item->search
> && ((item->expression_vars < 2 && is_node_table(item->table))
> || (item->expression_vars < 1))
> && !is_result_table(item->table))
>
> I have taken a look and cannot work out what it means.
In short, it's checking whether SQLite is using sufficiently many index
columns. STMT_SELECT_UNREFERENCED_PRISTINES does a table scan without
using an index, meaning that query does O(N) work where N is the number
of rows in the table.
To confirm that, does the test pass if you do «CREATE INDEX foo ON
pristine(refcount)» when creating the SQLite database the test runs on?
I haven't looked to see whether that's the right fix. That would depend
on whether STMT_SELECT_UNREFERENCED_PRISTINES needs to be performant,
and if it does, on what kind of index would best serve it without
affecting other queries (e.g., INSERTs and DELETEs to that table).
> Maybe the statement needs to be made to match the exception clause
> "in_list(primary_key_statements, i)", but that's just a semi-educated guess.
The statements in that list both have «LIMIT 1» on them. That'd be why
they're exempted from the "all queries must use an index" check.
>
Excuse brevity.
Daniel
Re: Double-checking 'pristines-on-demand-on-mwf' branch doc update.
Posted by Julian Foad <ju...@apache.org>.
Karl Fogel wrote:
> Julian, just FYI I committed r1898020 to update the
> 'BRANCH-README' file on the new 'pristines-on-demand-on-mwf'
> branch. [...]
Thanks for the heads-up. That's a useful clarification of the combined
README's on the combined branch.
- Julian
Double-checking 'pristines-on-demand-on-mwf' branch doc update.
Posted by Karl Fogel <kf...@red-bean.com>.
On 11 Feb 2022, Julian Foad wrote:
>Re. pristines-on-demand. I have re-based pristines-on-demand on
>top of
>multi-wc-format, resulting in a new branch
>'pristines-on-demand-on-mwf'.
Julian, just FYI I committed r1898020 to update the
'BRANCH-README' file on the new 'pristines-on-demand-on-mwf'
branch.
I'm *pretty* sure this was something you would have wanted and
that my commit is a) accurate and b) won't get in your way at all,
but I thought I'd double-check since you're more expert than I
about that branch.
Best regards,
-Karl