You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by "tballison (via GitHub)" <gi...@apache.org> on 2026/06/07 11:06:07 UTC
[PR] small updates to tika-eval [tika]
tballison opened a new pull request, #2881:
URL: https://github.com/apache/tika/pull/2881
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Thanks for your contribution to [Apache Tika](https://tika.apache.org/)! Your help is appreciated!
Before opening the pull request, please verify that
* there is an open issue on the [Tika issue tracker](https://issues.apache.org/jira/projects/TIKA) which describes the problem or the improvement. We cannot accept pull requests without an issue because the change wouldn't be listed in the release notes.
* the issue ID (`TIKA-XXXX`)
- is referenced in the title of the pull request
- and placed in front of your commit messages surrounded by square brackets (`[TIKA-XXXX] Issue or pull request title`)
* commits are squashed into a single one (or few commits for larger changes)
* Tika is successfully built and unit tests pass by running `./mvnw clean test`
* there should be no conflicts when merging the pull request branch into the *recent* `main` branch. If there are conflicts, please try to rebase the pull request branch on top of a freshly pulled `main` branch
* if you add new module that downstream users will depend upon add it to relevant group in `tika-bom/pom.xml`.
We will be able to faster integrate your pull request if these conditions are met. If you have any questions how to fix your problem or about using Tika in general, please sign up for the [Tika mailing list](http://tika.apache.org/mail-lists.html). Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] small updates to tika-eval [tika]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #2881:
URL: https://github.com/apache/tika/pull/2881#discussion_r3369989895
##########
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ExtractComparerRunner.java:
##########
@@ -131,12 +132,16 @@ public static void main(String[] args) throws Exception {
ResultsReporter.main(new String[]{"-d", dbPath, "-rd", reportsDir});
Path reportsDirPath = Paths.get(reportsDir);
if (Files.isDirectory(reportsDirPath)) {
- Path tgzPath = reportsDirPath.resolveSibling(reportsDir + ".tar.gz");
+ Path tgzPath = reportsDirPath.resolveSibling(reportsDirPath.getFileName() + ".tgz");
Review Comment:
When `-d` is provided as a full JDBC URL (supported by `getJdbcConnectionString`), the auto-report path still invokes `ResultsReporter` with `-d <dbPath>`. `ResultsReporter` treats `-d` as an H2 database name/path and will fail for JDBC URLs; it expects `-jdbc` for that case. This makes `-r` unusable with JDBC URLs even though Compare itself supports them.
##########
docs/modules/ROOT/pages/advanced/integration-testing/tika-eval-regression.adoc:
##########
@@ -203,9 +204,13 @@ Options:
tika-eval will create `\{label}.mv.db` and a `\{label}-reports/` dir
alongside. Persist the db if you want to re-run Report later.
* `-r` / `--report` — automatically run the Report step after Compare,
- and zip the reports directory.
+ and tgz the reports directory to `<reportsDir>.tgz`.
* `-rd` / `--reportsDir` — explicit reports directory (overrides the
default derived from `-d`).
+* `-z` / `--gzip` — gzip the H2 db file to `<db>.mv.db.gz` after Compare
+ so it can be transferred. Requires `-d` (no-op with a warning for a
+ temp db or a jdbc connection string). Combine with `-r` to package
+ both the reports and the db.
Review Comment:
The `-z/--gzip` flag is documented as a no-op for "a jdbc connection string", but the implementation actually supports `jdbc:h2:file:` URLs (it extracts the file path and gzips the resulting `.mv.db`). The docs should narrow this to *non-file* JDBC connections (e.g., `jdbc:h2:mem:` / tcp) to avoid misleading users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] small updates to tika-eval [tika]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #2881:
URL: https://github.com/apache/tika/pull/2881#discussion_r3369329069
##########
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ExtractComparerRunner.java:
##########
@@ -131,12 +132,16 @@ public static void main(String[] args) throws Exception {
ResultsReporter.main(new String[]{"-d", dbPath, "-rd", reportsDir});
Path reportsDirPath = Paths.get(reportsDir);
if (Files.isDirectory(reportsDirPath)) {
- Path tgzPath = reportsDirPath.resolveSibling(reportsDir + ".tar.gz");
+ Path tgzPath = reportsDirPath.resolveSibling(reportsDir + ".tgz");
LOG.info("Creating {}", tgzPath);
createTarGz(reportsDirPath, tgzPath);
Review Comment:
`resolveSibling(reportsDir + ".tgz")` can produce an incorrect path when `-rd/--reportsDir` contains path separators (e.g., `foo/bar` becomes `foo/foo/bar.tgz`). Use the reports directory path's filename for the sibling archive name instead.
##########
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ExtractComparerRunner.java:
##########
@@ -263,6 +268,40 @@ private static void deleteDirectory(Path dir) throws IOException {
}
}
+ /**
+ * Gzip the H2 db file (<dbPath>.mv.db -> <dbPath>.mv.db.gz) so it can be
+ * transferred. The db connection is already closed by {@link #execute} before
+ * this runs, so the file is unlocked. No-op (with a warning) when there is no
+ * on-disk file db to gzip: a temp db (no -d) or an explicit jdbc connection string.
+ */
+ private static void gzipDb(String dbPath, boolean usesTempDb) throws IOException {
+ if (usesTempDb) {
+ LOG.warn("-z (gzip) ignored: no -d db specified, so there is no db file to transfer");
+ return;
+ }
+ if (dbPath.startsWith("jdbc:")) {
+ LOG.warn("-z (gzip) ignored: db is an explicit jdbc connection ({}), not a local file", dbPath);
+ return;
+ }
+ Path dbFile = Paths.get(dbPath + ".mv.db");
+ if (!Files.isRegularFile(dbFile)) {
Review Comment:
`-z/--gzip` is ignored for any JDBC connection string, including `jdbc:h2:file:...` URLs that do point at an on-disk DB file. Since `-d` already accepts JDBC URLs, consider supporting the common `jdbc:h2:file:` case by extracting the file base path (stripping any `;` options) instead of unconditionally warning and returning.
##########
docs/modules/ROOT/pages/advanced/integration-testing/tika-eval-regression.adoc:
##########
@@ -191,7 +191,8 @@ java -jar tika-eval/tika-eval-app/target/tika-eval-app-{tika-version}.jar \
The `Compare` subcommand keyword is optional — the CLI infers it from
the `-a` / `-b` flags. The `-r` flag both runs the Report stage and
-zips the resulting reports directory for easy archiving.
+tgz's the resulting reports directory (`<reportsDir>.tgz`) for easy
+archiving.
Review Comment:
“tgz's” reads informal/awkward in docs. Consider rephrasing to describe creating a `.tgz` archive instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] small updates to tika-eval [tika]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #2881:
URL: https://github.com/apache/tika/pull/2881#discussion_r3380657769
##########
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ExtractComparerRunner.java:
##########
@@ -263,6 +272,53 @@ private static void deleteDirectory(Path dir) throws IOException {
}
}
+ /**
+ * Gzip the H2 db file (<dbPath>.mv.db -> <dbPath>.mv.db.gz) so it can be
+ * transferred. The db connection is already closed by {@link #execute} before
+ * this runs, so the file is unlocked. No-op (with a warning) when there is no
+ * on-disk file db to gzip: a temp db (no -d), or a non-file jdbc connection
+ * (e.g. mem/tcp). A {@code jdbc:h2:file:} URL is supported by extracting its
+ * file path.
+ */
+ private static void gzipDb(String dbPath, boolean usesTempDb) throws IOException {
+ if (usesTempDb) {
+ LOG.warn("-z (gzip) ignored: no -d db specified, so there is no db file to transfer");
+ return;
+ }
+ String filePath = dbPath;
+ if (dbPath.startsWith("jdbc:")) {
+ String prefix = "jdbc:h2:file:";
+ if (!dbPath.startsWith(prefix)) {
+ LOG.warn("-z (gzip) ignored: db is a non-file jdbc connection ({}), no local file to transfer",
+ dbPath);
+ return;
+ }
+ // Strip the jdbc:h2:file: prefix and any ;OPTION=... suffix to get the file base path.
+ filePath = dbPath.substring(prefix.length());
+ int semi = filePath.indexOf(';');
+ if (semi >= 0) {
+ filePath = filePath.substring(0, semi);
+ }
+ }
Review Comment:
`gzipDb` only treats `jdbc:h2:file:` URLs as file-backed and will incorrectly warn/skip for valid file-based H2 URLs like `jdbc:h2:/path/to/db` (which is also what `H2Util` generates). It also doesn't handle callers passing `-d ...mv.db`, which would make it look for `...mv.db.mv.db`. Consider supporting the broader H2 file URL forms and trimming an existing `.mv.db` suffix before appending it again.
##########
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ExtractComparerRunner.java:
##########
@@ -128,15 +129,23 @@ public static void main(String[] args) throws Exception {
if (commandLine.hasOption('r')) {
String reportsDir = commandLine.getOptionValue("rd", "reports");
LOG.info("Running Report...");
- ResultsReporter.main(new String[]{"-d", dbPath, "-rd", reportsDir});
+ if (dbPath.startsWith("jdbc:")) {
+ ResultsReporter.main(new String[]{"-jdbc", dbPath, "-rd", reportsDir});
+ } else {
+ ResultsReporter.main(new String[]{"-d", dbPath, "-rd", reportsDir});
+ }
Path reportsDirPath = Paths.get(reportsDir);
if (Files.isDirectory(reportsDirPath)) {
- Path tgzPath = reportsDirPath.resolveSibling(reportsDir + ".tar.gz");
+ Path tgzPath = reportsDirPath.resolveSibling(reportsDirPath.getFileName() + ".tgz");
Review Comment:
`reportsDirPath.getFileName()` can return null when `reportsDirPath` is the filesystem root (e.g., `-rd /` on *nix), which would cause a NullPointerException when building the archive name. Use a null-safe basename when constructing the `.tgz` path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] small updates to tika-eval [tika]
Posted by "tballison (via GitHub)" <gi...@apache.org>.
tballison merged PR #2881:
URL: https://github.com/apache/tika/pull/2881
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org