You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tika.apache.org by ta...@apache.org on 2026/08/12 16:15:56 UTC
(tika) branch TIKA-4816-metadata-key-api created (now 0e3fe95059)
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a change to branch TIKA-4816-metadata-key-api
in repository https://gitbox.apache.org/repos/asf/tika.git
at 0e3fe95059 TIKA-4816 fix EMBEDDED_EXCEPTION erasure under active metadata limiter
This branch includes the following new commits:
new eaa0847897 TIKA-4816 metadata-key stage 1: Property registration fix
new ff467d33c5 TIKA-4816 metadata-key stage 2: mint-time reserved-name validation
new bb60edb384 TIKA-4816 metadata-key stage 3: KeyPrefix rename + constructor validation + typed minting factories
new 24606fcd3b TIKA-4816 metadata-key stage 4: putAll(Metadata) public copy API
new 6acacb95e4 TIKA-4816 metadata-key stage 8: remove setAll, promote ClimateForcast (+strays), docs reconciliation
new bd803d799c TIKA-4816 metadata-key stage 9: reconstruct negative-path tests + migration guide
new 0e3fe95059 TIKA-4816 fix EMBEDDED_EXCEPTION erasure under active metadata limiter
The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(tika) 05/07: TIKA-4816 metadata-key stage 8: remove setAll, promote ClimateForcast (+strays), docs reconciliation
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4816-metadata-key-api
in repository https://gitbox.apache.org/repos/asf/tika.git
commit 6acacb95e4b10f72a64a0cc93115c859f2574515
Author: tallison <ta...@apache.org>
AuthorDate: Tue Aug 11 15:36:22 2026 -0400
TIKA-4816 metadata-key stage 8: remove setAll, promote ClimateForcast (+strays), docs reconciliation
---
.skills/metadata-schema.md | 3 ++
.../ROOT/pages/developers/metadata-keys.adoc | 9 ++++--
.../org/apache/tika/metadata/ClimateForcast.java | 34 ++++++++++++----------
.../org/apache/tika/metadata/CreativeCommons.java | 6 ++--
.../java/org/apache/tika/metadata/Metadata.java | 18 +-----------
.../org/apache/tika/metadata/TikaMimeKeys.java | 28 ------------------
.../org/apache/tika/metadata/TestMetadata.java | 31 --------------------
.../apache/tika/metadata/metadata-key-fields.json | 18 ++++++++++++
.../org/apache/tika/metadata/metadata-keys.json | 18 ++++++++++++
.../metadata/schema/MetadataNoUnderscoreTest.java | 12 ++++++--
.../org/apache/tika/parser/grib/GribParser.java | 25 +++++++++++-----
.../apache/tika/parser/netcdf/NetCDFParser.java | 25 +++++++++++-----
.../main/java/org/apache/tika/xmp/XMPMetadata.java | 9 +++---
13 files changed, 116 insertions(+), 120 deletions(-)
diff --git a/.skills/metadata-schema.md b/.skills/metadata-schema.md
index a31c02cc74..88a1351d00 100644
--- a/.skills/metadata-schema.md
+++ b/.skills/metadata-schema.md
@@ -65,6 +65,9 @@ Failures with stale `X-TIKA:`/underscore/`SHA256` keys usually mean *regenerate*
## Naming conventions (frozen for 4.0, TIKA-4794)
- All keys are `Property` constants — no bare `String` keys (`metadata-string-keys.json` retired).
+ Exception: a few deprecated `String` fields (`IPTC.*_WRONG_CASE`,
+ `TikaCoreProperties.EMBEDDED_RESOURCE_TYPE_KEY`) exist only to construct a real `Property`'s
+ name and aren't independent keys.
- Tika-coined prefix is `tk:` (`X-TIKA:` is legacy); kebab-case, no underscores.
- External-standard names verbatim, *including* the standard's prefix: `dc:`, `xmp:`, `cp:`, `extended-properties:`.
- HTTP has no namespace → `Content-Type`, `Content-Encoding`, `Location` stay bare (no `http:`).
diff --git a/docs/modules/ROOT/pages/developers/metadata-keys.adoc b/docs/modules/ROOT/pages/developers/metadata-keys.adoc
index 71ca570b70..90fd19ed1b 100644
--- a/docs/modules/ROOT/pages/developers/metadata-keys.adoc
+++ b/docs/modules/ROOT/pages/developers/metadata-keys.adoc
@@ -16,9 +16,12 @@
= Adding a Metadata Key
Every metadata key Tika can emit is a `Property` constant (or, for runtime-minted names like scraped
-HTML `<meta>` tags, a `KeyPrefix`) — there are no bare `String` keys. That closed/open key
-space is tracked in a generated, build-gated registry, so adding a key involves one extra step beyond
-writing the Java.
+HTML `<meta>` tags, a `KeyPrefix`) — there are no bare `String` keys naming a population of writable
+metadata. (A handful of `String` constants remain, e.g. `IPTC`'s four `*_WRONG_CASE` fields and
+`TikaCoreProperties.EMBEDDED_RESOURCE_TYPE_KEY`; these are deprecated/internal building blocks that
+construct a real `Property`'s name — not independent keys — and are excluded from the registry's key
+count for that reason.) That closed/open key space is tracked in a generated, build-gated registry, so
+adding a key involves one extra step beyond writing the Java.
== Add the constant
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/ClimateForcast.java b/tika-core/src/main/java/org/apache/tika/metadata/ClimateForcast.java
index 9ad1632837..ccffa8c980 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/ClimateForcast.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/ClimateForcast.java
@@ -18,38 +18,40 @@ package org.apache.tika.metadata;
/**
* Met keys from NCAR CCSM files in the <a
- * href="http://cf-pcmdi.llnl.gov/">Climate Forecast Convention</a>.
+ * href="http://cf-pcmdi.llnl.gov/">Climate Forecast Convention</a>. Key spellings are the
+ * external convention's own attribute names, verbatim (not Tika-coined) — some contain
+ * underscores per that convention.
*/
public interface ClimateForcast {
- String PROGRAM_ID = "prg_ID";
+ Property PROGRAM_ID = Property.externalText("prg_ID");
- String COMMAND_LINE = "cmd_ln";
+ Property COMMAND_LINE = Property.externalText("cmd_ln");
- String HISTORY = "history";
+ Property HISTORY = Property.externalText("history");
- String TABLE_ID = "table_id";
+ Property TABLE_ID = Property.externalText("table_id");
- String INSTITUTION = "institution";
+ Property INSTITUTION = Property.externalText("institution");
- String SOURCE = "source";
+ Property SOURCE = Property.externalText("source");
- String CONTACT = "contact";
+ Property CONTACT = Property.externalText("contact");
- String PROJECT_ID = "project_id";
+ Property PROJECT_ID = Property.externalText("project_id");
- String CONVENTIONS = "Conventions";
+ Property CONVENTIONS = Property.externalText("Conventions");
- String REFERENCES = "references";
+ Property REFERENCES = Property.externalText("references");
- String ACKNOWLEDGEMENT = "acknowledgement";
+ Property ACKNOWLEDGEMENT = Property.externalText("acknowledgement");
- String REALIZATION = "realization";
+ Property REALIZATION = Property.externalText("realization");
- String EXPERIMENT_ID = "experiment_id";
+ Property EXPERIMENT_ID = Property.externalText("experiment_id");
- String COMMENT = "comment";
+ Property COMMENT = Property.externalText("comment");
- String MODEL_NAME_ENGLISH = "model_name_english";
+ Property MODEL_NAME_ENGLISH = Property.externalText("model_name_english");
}
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/CreativeCommons.java b/tika-core/src/main/java/org/apache/tika/metadata/CreativeCommons.java
index 122a1fc578..2491d5e1a6 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/CreativeCommons.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/CreativeCommons.java
@@ -23,10 +23,10 @@ package org.apache.tika.metadata;
*/
public interface CreativeCommons {
- String LICENSE_URL = "License-Url";
+ Property LICENSE_URL = Property.externalText("License-Url");
- String LICENSE_LOCATION = "License-Location";
+ Property LICENSE_LOCATION = Property.externalText("License-Location");
- String WORK_TYPE = "Work-Type";
+ Property WORK_TYPE = Property.externalText("Work-Type");
}
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/Metadata.java b/tika-core/src/main/java/org/apache/tika/metadata/Metadata.java
index 9ea8385850..2a6dd56a1e 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/Metadata.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/Metadata.java
@@ -25,13 +25,11 @@ import java.text.SimpleDateFormat;
import java.util.Arrays;
import java.util.Calendar;
import java.util.Date;
-import java.util.Enumeration;
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Objects;
-import java.util.Properties;
import java.util.TimeZone;
import org.slf4j.Logger;
@@ -48,7 +46,7 @@ import org.apache.tika.utils.DateUtils;
*/
public class Metadata
implements CreativeCommons, Geographic, HttpHeaders, Message, ClimateForcast, TIFF,
- TikaMimeKeys, Serializable {
+ Serializable {
private static final Logger LOG = LoggerFactory.getLogger(Metadata.class);
@@ -448,20 +446,6 @@ public class Metadata
}
}
- /**
- * Copy All key-value pairs from properties.
- *
- * @param properties properties to copy from
- */
- @SuppressWarnings("unchecked")
- public void setAll(Properties properties) {
- Enumeration<String> names = (Enumeration<String>) properties.propertyNames();
- while (names.hasMoreElements()) {
- String name = names.nextElement();
- metadata.put(name, new String[]{properties.getProperty(name)});
- }
- }
-
/**
* Set metadata name/value. Associate the specified value to the specified
* metadata name. If some previous values were associated to this name,
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/TikaMimeKeys.java b/tika-core/src/main/java/org/apache/tika/metadata/TikaMimeKeys.java
deleted file mode 100644
index 7ae685e05e..0000000000
--- a/tika-core/src/main/java/org/apache/tika/metadata/TikaMimeKeys.java
+++ /dev/null
@@ -1,28 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.tika.metadata;
-
-/**
- * A collection of Tika metadata keys used in Mime Type resolution
- */
-public interface TikaMimeKeys {
-
- String TIKA_MIME_FILE = "tika.mime.file";
-
- String MIME_TYPE_MAGIC = "mime.type.magic";
-
-}
diff --git a/tika-core/src/test/java/org/apache/tika/metadata/TestMetadata.java b/tika-core/src/test/java/org/apache/tika/metadata/TestMetadata.java
index 2ce1b8b6c6..e92fe4c3f0 100644
--- a/tika-core/src/test/java/org/apache/tika/metadata/TestMetadata.java
+++ b/tika-core/src/test/java/org/apache/tika/metadata/TestMetadata.java
@@ -27,7 +27,6 @@ import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Locale;
-import java.util.Properties;
import java.util.Random;
import java.util.TimeZone;
import java.util.concurrent.Callable;
@@ -120,36 +119,6 @@ public class TestMetadata extends TikaTest {
assertEquals("new value 2", values[1]);
}
- /**
- * Test for <code>setAll(Properties)</code> method.
- */
- @Test
- public void testSetProperties() {
- String[] values = null;
- Metadata meta = new Metadata();
- Properties props = new Properties();
-
- meta.setAll(props);
- assertEquals(0, meta.size());
-
- props.setProperty("name-one", "value1.1");
- meta.setAll(props);
- assertEquals(1, meta.size());
- values = meta.getValues("name-one");
- assertEquals(1, values.length);
- assertEquals("value1.1", values[0]);
-
- props.setProperty("name-two", "value2.1");
- meta.setAll(props);
- assertEquals(2, meta.size());
- values = meta.getValues("name-one");
- assertEquals(1, values.length);
- assertEquals("value1.1", values[0]);
- values = meta.getValues("name-two");
- assertEquals(1, values.length);
- assertEquals("value2.1", values[0]);
- }
-
/**
* Test for <code>get(String)</code> method.
*/
diff --git a/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-key-fields.json b/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-key-fields.json
index 94488c9af7..a778b9909b 100644
--- a/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-key-fields.json
+++ b/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-key-fields.json
@@ -17,6 +17,24 @@
{"class":"org.apache.tika.metadata.Audio","field":"RAW_DISC_NUMBER","key":"audio:raw-disc-number"},
{"class":"org.apache.tika.metadata.Audio","field":"RAW_TRACK_NUMBER","key":"audio:raw-track-number"},
{"class":"org.apache.tika.metadata.Audio","field":"TRACK_COUNT","key":"audio:track-count"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"ACKNOWLEDGEMENT","key":"acknowledgement"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"COMMAND_LINE","key":"cmd_ln"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"COMMENT","key":"comment"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"CONTACT","key":"contact"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"CONVENTIONS","key":"Conventions"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"EXPERIMENT_ID","key":"experiment_id"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"HISTORY","key":"history"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"INSTITUTION","key":"institution"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"MODEL_NAME_ENGLISH","key":"model_name_english"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"PROGRAM_ID","key":"prg_ID"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"PROJECT_ID","key":"project_id"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"REALIZATION","key":"realization"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"REFERENCES","key":"references"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"SOURCE","key":"source"},
+ {"class":"org.apache.tika.metadata.ClimateForcast","field":"TABLE_ID","key":"table_id"},
+ {"class":"org.apache.tika.metadata.CreativeCommons","field":"LICENSE_LOCATION","key":"License-Location"},
+ {"class":"org.apache.tika.metadata.CreativeCommons","field":"LICENSE_URL","key":"License-Url"},
+ {"class":"org.apache.tika.metadata.CreativeCommons","field":"WORK_TYPE","key":"Work-Type"},
{"class":"org.apache.tika.metadata.DWG","field":"APPLICATION_COMMENT","key":"dwg:application-comment"},
{"class":"org.apache.tika.metadata.DWG","field":"APPLICATION_NAME","key":"dwg:application-name"},
{"class":"org.apache.tika.metadata.DWG","field":"APPLICATION_VERSION","key":"dwg:application-version"},
diff --git a/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-keys.json b/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-keys.json
index f601ddfc8f..4297dd26ef 100644
--- a/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-keys.json
+++ b/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-keys.json
@@ -13,6 +13,7 @@
{"key":"Content-Location","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"Content-MD5","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"Content-Type","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"Conventions","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"Iptc4xmpCore:CiAdrCity","namespace":"Iptc4xmpCore","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"Iptc4xmpCore:CiAdrCtry","namespace":"Iptc4xmpCore","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"Iptc4xmpCore:CiAdrExtadr","namespace":"Iptc4xmpCore","valueType":"TEXT","cardinality":"BAG"},
@@ -64,7 +65,10 @@
{"key":"Iptc4xmpExt:RegItemId","namespace":"Iptc4xmpExt","valueType":"TEXT","cardinality":"BAG"},
{"key":"Iptc4xmpExt:RegOrgId","namespace":"Iptc4xmpExt","valueType":"TEXT","cardinality":"BAG"},
{"key":"Iptc4xmpExt:RegistryId","namespace":"Iptc4xmpExt","valueType":"TEXT","cardinality":"BAG"},
+ {"key":"License-Location","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"License-Url","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"Location","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"Work-Type","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"access-permission:assemble-document","namespace":"access-permission","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"access-permission:can-modify","namespace":"access-permission","valueType":"TEXT","cardinality":"BAG"},
{"key":"access-permission:can-print","namespace":"access-permission","valueType":"TEXT","cardinality":"SIMPLE"},
@@ -73,6 +77,7 @@
{"key":"access-permission:extract-for-accessibility","namespace":"access-permission","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"access-permission:fill-in-form","namespace":"access-permission","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"access-permission:modify-annotations","namespace":"access-permission","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"acknowledgement","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"audio:bitrate","namespace":"audio","valueType":"INTEGER","cardinality":"SIMPLE"},
{"key":"audio:bits-per-sample","namespace":"audio","valueType":"INTEGER","cardinality":"SIMPLE"},
{"key":"audio:channels","namespace":"audio","valueType":"INTEGER","cardinality":"SIMPLE"},
@@ -86,6 +91,9 @@
{"key":"aux:LensID","namespace":"aux","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"aux:LensInfo","namespace":"aux","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"aux:SerialNumber","namespace":"aux","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"cmd_ln","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"comment","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"contact","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"cp:category","namespace":"cp","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"cp:contentStatus","namespace":"cp","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"cp:lastModifiedBy","namespace":"cp","valueType":"TEXT","cardinality":"SIMPLE"},
@@ -132,6 +140,7 @@
{"key":"exif:FocalLength","namespace":"exif","valueType":"RATIONAL","cardinality":"SIMPLE"},
{"key":"exif:IsoSpeedRatings","namespace":"exif","valueType":"INTEGER","cardinality":"SEQ"},
{"key":"exif:PageCount","namespace":"exif","valueType":"INTEGER","cardinality":"SIMPLE"},
+ {"key":"experiment_id","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"extended-properties:AppVersion","namespace":"extended-properties","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"extended-properties:Application","namespace":"extended-properties","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"extended-properties:Company","namespace":"extended-properties","valueType":"TEXT","cardinality":"SIMPLE"},
@@ -160,7 +169,9 @@
{"key":"geo:lat","namespace":"geo","valueType":"REAL","cardinality":"SIMPLE"},
{"key":"geo:long","namespace":"geo","valueType":"REAL","cardinality":"SIMPLE"},
{"key":"geo:timestamp","namespace":"geo","valueType":"DATE","cardinality":"SIMPLE"},
+ {"key":"history","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"html:scriptSrc","namespace":"html","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"institution","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"iworks:build-version-history","namespace":"iworks","valueType":"TEXT","cardinality":"BAG"},
{"key":"iworks:document-id","namespace":"iworks","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"machine:architectureBits","namespace":"machine","valueType":"CLOSED_CHOICE","cardinality":"SIMPLE"},
@@ -244,6 +255,7 @@
{"key":"meta:slide-count","namespace":"meta","valueType":"INTEGER","cardinality":"SIMPLE"},
{"key":"meta:table-count","namespace":"meta","valueType":"INTEGER","cardinality":"SIMPLE"},
{"key":"meta:word-count","namespace":"meta","valueType":"INTEGER","cardinality":"SIMPLE"},
+ {"key":"model_name_english","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"msoffice:comment-person-display-name","namespace":"msoffice","valueType":"TEXT","cardinality":"BAG"},
{"key":"msoffice:doc:has-attached-template","namespace":"msoffice","valueType":"BOOLEAN","cardinality":"SIMPLE"},
{"key":"msoffice:doc:has-framesets","namespace":"msoffice","valueType":"BOOLEAN","cardinality":"SIMPLE"},
@@ -386,6 +398,8 @@
{"key":"plus:PropertyReleaseID","namespace":"plus","valueType":"TEXT","cardinality":"BAG"},
{"key":"plus:PropertyReleaseStatus","namespace":"plus","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"plus:Version","namespace":"plus","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"prg_ID","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"project_id","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"pst:discriptor-node-id","namespace":"pst","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"pst:is-valid","namespace":"pst","valueType":"BOOLEAN","cardinality":"SIMPLE"},
{"key":"quattropro:build","namespace":"quattropro","valueType":"INTEGER","cardinality":"SIMPLE"},
@@ -393,12 +407,16 @@
{"key":"quattropro:lowest-version","namespace":"quattropro","valueType":"INTEGER","cardinality":"SIMPLE"},
{"key":"quattropro:version","namespace":"quattropro","valueType":"INTEGER","cardinality":"SIMPLE"},
{"key":"quicktime:still-image-time","namespace":"quicktime","valueType":"REAL","cardinality":"SIMPLE"},
+ {"key":"realization","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"references","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"rtf:contains-encapsulated-html","namespace":"rtf","valueType":"BOOLEAN","cardinality":"SIMPLE"},
{"key":"rtf:embedded-app-version","namespace":"rtf","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"rtf:embedded-class","namespace":"rtf","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"rtf:embedded-item","namespace":"rtf","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"rtf:embedded-topic","namespace":"rtf","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"rtf:thumbnail","namespace":"rtf","valueType":"BOOLEAN","cardinality":"SIMPLE"},
+ {"key":"source","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
+ {"key":"table_id","namespace":"","valueType":"TEXT","cardinality":"SIMPLE"},
{"key":"tess:image-magick-processed","namespace":"tess","valueType":"BOOLEAN","cardinality":"SEQ"},
{"key":"tess:orientation","namespace":"tess","valueType":"INTEGER","cardinality":"SIMPLE"},
{"key":"tess:orientation-confidence","namespace":"tess","valueType":"REAL","cardinality":"SIMPLE"},
diff --git a/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataNoUnderscoreTest.java b/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataNoUnderscoreTest.java
index 7ee7600fab..ee799d7d0f 100644
--- a/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataNoUnderscoreTest.java
+++ b/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataNoUnderscoreTest.java
@@ -34,8 +34,16 @@ import org.junit.jupiter.api.Test;
*/
public class MetadataNoUnderscoreTest {
- /** Keys/prefixes whose underscore is an external standard's spelling (verbatim). Empty today. */
- private static final Set<String> ALLOWLIST = Set.of();
+ /**
+ * Keys/prefixes whose underscore is an external standard's spelling (verbatim).
+ * {@code ClimateForcast} (tika-core) mints its 15 constants from the NCAR CCSM /
+ * Climate Forecast convention (http://cf-pcmdi.llnl.gov/) attribute names as-is; 6 of
+ * them carry the convention's own underscores. These are not Tika-coined, so renaming
+ * them is out of scope here (it would also change the keys Tika has emitted since 0.x).
+ */
+ private static final Set<String> ALLOWLIST = Set.of(
+ "prg_ID", "cmd_ln", "table_id", "project_id", "experiment_id",
+ "model_name_english");
@Test
public void noUnderscoreInTikaCoinedKeys() throws Exception {
diff --git a/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/grib/GribParser.java b/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/grib/GribParser.java
index 94f819f9f2..2649b0dd20 100644
--- a/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/grib/GribParser.java
+++ b/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/grib/GribParser.java
@@ -21,7 +21,10 @@ import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.util.Collections;
+import java.util.Map;
import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
import org.apache.commons.io.FileUtils;
import org.xml.sax.ContentHandler;
@@ -38,6 +41,7 @@ import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.ClimateForcast;
import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
+import org.apache.tika.metadata.Property;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
@@ -124,18 +128,23 @@ public class GribParser implements Parser {
}
}
- private static final Set<String> CF_GLOBAL_ATTRIBUTES = Set.of(
- ClimateForcast.PROGRAM_ID, ClimateForcast.COMMAND_LINE, ClimateForcast.HISTORY,
- ClimateForcast.TABLE_ID, ClimateForcast.INSTITUTION, ClimateForcast.SOURCE,
- ClimateForcast.CONTACT, ClimateForcast.PROJECT_ID, ClimateForcast.CONVENTIONS,
- ClimateForcast.REFERENCES, ClimateForcast.ACKNOWLEDGEMENT, ClimateForcast.REALIZATION,
- ClimateForcast.EXPERIMENT_ID, ClimateForcast.COMMENT, ClimateForcast.MODEL_NAME_ENGLISH);
+ private static final Map<String, Property> CF_GLOBAL_ATTRIBUTES = Stream.of(
+ ClimateForcast.PROGRAM_ID, ClimateForcast.COMMAND_LINE, ClimateForcast.HISTORY,
+ ClimateForcast.TABLE_ID, ClimateForcast.INSTITUTION, ClimateForcast.SOURCE,
+ ClimateForcast.CONTACT, ClimateForcast.PROJECT_ID, ClimateForcast.CONVENTIONS,
+ ClimateForcast.REFERENCES, ClimateForcast.ACKNOWLEDGEMENT,
+ ClimateForcast.REALIZATION, ClimateForcast.EXPERIMENT_ID, ClimateForcast.COMMENT,
+ ClimateForcast.MODEL_NAME_ENGLISH)
+ .collect(Collectors.toMap(Property::getName, p -> p));
private static void addGlobalAttribute(Metadata metadata, String name, String value) {
if ("title".equals(name)) {
metadata.add(TikaCoreProperties.TITLE, value);
- } else if (CF_GLOBAL_ATTRIBUTES.contains(name)) {
- metadata.add(name, value);
+ return;
+ }
+ Property cfProperty = CF_GLOBAL_ATTRIBUTES.get(name);
+ if (cfProperty != null) {
+ metadata.add(cfProperty, value);
} else {
metadata.add(GRIB.key(name), value);
}
diff --git a/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/netcdf/NetCDFParser.java b/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/netcdf/NetCDFParser.java
index 1cc2cef3e2..364e7cd522 100644
--- a/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/netcdf/NetCDFParser.java
+++ b/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/netcdf/NetCDFParser.java
@@ -21,7 +21,10 @@ package org.apache.tika.parser.netcdf;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
+import java.util.Map;
import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
import org.xml.sax.ContentHandler;
import org.xml.sax.SAXException;
@@ -36,6 +39,7 @@ import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.ClimateForcast;
import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
+import org.apache.tika.metadata.Property;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
@@ -133,18 +137,23 @@ public class NetCDFParser implements Parser {
}
}
- private static final Set<String> CF_GLOBAL_ATTRIBUTES = Set.of(
- ClimateForcast.PROGRAM_ID, ClimateForcast.COMMAND_LINE, ClimateForcast.HISTORY,
- ClimateForcast.TABLE_ID, ClimateForcast.INSTITUTION, ClimateForcast.SOURCE,
- ClimateForcast.CONTACT, ClimateForcast.PROJECT_ID, ClimateForcast.CONVENTIONS,
- ClimateForcast.REFERENCES, ClimateForcast.ACKNOWLEDGEMENT, ClimateForcast.REALIZATION,
- ClimateForcast.EXPERIMENT_ID, ClimateForcast.COMMENT, ClimateForcast.MODEL_NAME_ENGLISH);
+ private static final Map<String, Property> CF_GLOBAL_ATTRIBUTES = Stream.of(
+ ClimateForcast.PROGRAM_ID, ClimateForcast.COMMAND_LINE, ClimateForcast.HISTORY,
+ ClimateForcast.TABLE_ID, ClimateForcast.INSTITUTION, ClimateForcast.SOURCE,
+ ClimateForcast.CONTACT, ClimateForcast.PROJECT_ID, ClimateForcast.CONVENTIONS,
+ ClimateForcast.REFERENCES, ClimateForcast.ACKNOWLEDGEMENT,
+ ClimateForcast.REALIZATION, ClimateForcast.EXPERIMENT_ID, ClimateForcast.COMMENT,
+ ClimateForcast.MODEL_NAME_ENGLISH)
+ .collect(Collectors.toMap(Property::getName, p -> p));
private static void addGlobalAttribute(Metadata metadata, String name, String value) {
if ("title".equals(name)) {
metadata.add(TikaCoreProperties.TITLE, value);
- } else if (CF_GLOBAL_ATTRIBUTES.contains(name)) {
- metadata.add(name, value);
+ return;
+ }
+ Property cfProperty = CF_GLOBAL_ATTRIBUTES.get(name);
+ if (cfProperty != null) {
+ metadata.add(cfProperty, value);
} else {
metadata.add(NETCDF.key(name), value);
}
diff --git a/tika-xmp/src/main/java/org/apache/tika/xmp/XMPMetadata.java b/tika-xmp/src/main/java/org/apache/tika/xmp/XMPMetadata.java
index 888ca95e71..a89fa2cfbb 100644
--- a/tika-xmp/src/main/java/org/apache/tika/xmp/XMPMetadata.java
+++ b/tika-xmp/src/main/java/org/apache/tika/xmp/XMPMetadata.java
@@ -484,11 +484,12 @@ public class XMPMetadata extends Metadata {
}
/**
- * It will set all simple and array properties that have QName keys in registered namespaces.
- *
- * @see org.apache.tika.metadata.Metadata#setAll(java.util.Properties)
+ * Sets all simple and array properties that have QName keys in registered namespaces.
+ * XMPMetadata-specific: no longer overrides a parent method (the raw-map
+ * {@code Metadata#setAll(Properties)} it used to override was removed as an
+ * unguarded write route; this method is a full reimplementation and never
+ * called {@code super}, so removal doesn't change its behavior).
*/
- @Override
public void setAll(Properties properties) {
@SuppressWarnings("unchecked") Enumeration<String> names =
(Enumeration<String>) properties.propertyNames();
(tika) 03/07: TIKA-4816 metadata-key stage 3: KeyPrefix rename + constructor validation + typed minting factories
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4816-metadata-key-api
in repository https://gitbox.apache.org/repos/asf/tika.git
commit bb60edb384a9b0ce2ad41b7649e8af0d5ee29a70
Author: tallison <ta...@apache.org>
AuthorDate: Tue Aug 11 13:40:27 2026 -0400
TIKA-4816 metadata-key stage 3: KeyPrefix rename + constructor validation + typed minting factories
---
.skills/metadata-schema.md | 6 +-
docs/modules/ROOT/pages/developers/index.adoc | 2 +-
.../ROOT/pages/developers/metadata-keys.adoc | 6 +-
.../main/java/org/apache/tika/metadata/DWG.java | 4 +-
.../main/java/org/apache/tika/metadata/HTML.java | 2 +-
.../java/org/apache/tika/metadata/KeyPrefix.java | 172 +++++++++++++++++++
.../main/java/org/apache/tika/metadata/MAPI.java | 2 +-
.../java/org/apache/tika/metadata/Message.java | 2 +-
.../main/java/org/apache/tika/metadata/Office.java | 2 +-
.../main/java/org/apache/tika/metadata/PDF.java | 4 +-
.../apache/tika/metadata/PassthroughPrefix.java | 79 ---------
.../java/org/apache/tika/metadata/RTFMetadata.java | 2 +-
.../org/apache/tika/metadata/KeyPrefixTest.java | 182 +++++++++++++++++++++
tika-metadata-schema/README.md | 6 +-
tika-metadata-schema/regen.sh | 8 +-
.../tika/metadata/schema/SchemaGenerator.java | 12 +-
.../tika/metadata/schema/MetadataCoverageTest.java | 4 +-
.../tika/metadata/schema/MetadataSchemaTest.java | 4 +-
.../apache/tika/parser/envi/EnviHeaderParser.java | 6 +-
.../org/apache/tika/parser/grib/GribParser.java | 6 +-
.../apache/tika/parser/netcdf/NetCDFParser.java | 6 +-
.../tika/parser/ctakes/CTAKESContentHandler.java | 6 +-
.../tika/parser/journal/GrobidRESTParser.java | 6 +-
.../apache/tika/parser/ner/NamedEntityParser.java | 6 +-
.../tika/parser/iwork/NumbersContentHandler.java | 6 +-
.../parser/iwork/iwana/IWork13PackageParser.java | 6 +-
.../org/apache/tika/parser/ogg/OggAudioParser.java | 6 +-
.../org/apache/tika/parser/video/FLVParser.java | 6 +-
.../java/org/apache/tika/parser/dwg/DWGParser.java | 6 +-
.../tika/parser/image/ImageMetadataExtractor.java | 10 +-
.../org/apache/tika/parser/mbox/MboxParser.java | 6 +-
.../tika/parser/microsoft/JackcessParser.java | 14 +-
.../org/apache/tika/parser/warc/WARCParser.java | 10 +-
.../org/apache/tika/parser/xmp/XmpExtractor.java | 4 +-
34 files changed, 442 insertions(+), 167 deletions(-)
diff --git a/.skills/metadata-schema.md b/.skills/metadata-schema.md
index da51bf1fdb..a31c02cc74 100644
--- a/.skills/metadata-schema.md
+++ b/.skills/metadata-schema.md
@@ -9,19 +9,19 @@ this covers conventions, regeneration, and the traps.
Under `tika-metadata-schema/src/main/resources/org/apache/tika/metadata/`:
- `metadata-keys.json` — closed set: every `Property` constant + the synthesized `tk:digest:*` cross-product.
-- `metadata-open-namespaces.json` — `PassthroughPrefix` prefixes for runtime-minted names (`html:`, `message:raw-header:`, `mdb-prop:`).
+- `metadata-open-namespaces.json` — `KeyPrefix` prefixes for runtime-minted names (`html:`, `message:raw-header:`, `mdb-prop:`).
- `metadata-key-fields.json` — TIKA-4797 `{class, field, key}` table for field-identity migration.
Committed on purpose: they are the reviewable audit trail of the key space (a rename or a dropped key
shows up as a diff). Don't switch to build-time-only generation — that loses the review signal.
-## Regenerate (after adding/changing a Property or PassthroughPrefix)
+## Regenerate (after adding/changing a Property or KeyPrefix)
```bash
tika-metadata-schema/regen.sh
```
-This does the full sequence in one shot: `-am install` so newly added Property/PassthroughPrefix
+This does the full sequence in one shot: `-am install` so newly added Property/KeyPrefix
classes are on the scan classpath, regenerate all three registries via the forked-exec profile, print
a before/after key-count check (catches an incomplete classpath scan), `git diff --stat` the
registries, then run the gate tests. Flags: `--skip-install` (only safe if nothing outside
diff --git a/docs/modules/ROOT/pages/developers/index.adoc b/docs/modules/ROOT/pages/developers/index.adoc
index ac1ef574be..95701969c6 100644
--- a/docs/modules/ROOT/pages/developers/index.adoc
+++ b/docs/modules/ROOT/pages/developers/index.adoc
@@ -22,7 +22,7 @@ with custom parsers, detectors, and other components.
* xref:developers/serialization.adoc[Serialization and Configuration] - JSON configuration,
@TikaComponent annotation, and creating custom components
-* xref:developers/metadata-keys.adoc[Adding a Metadata Key] - the Property/PassthroughPrefix
+* xref:developers/metadata-keys.adoc[Adding a Metadata Key] - the Property/KeyPrefix
registry, naming conventions, and regenerating the schema
== Coming Soon
diff --git a/docs/modules/ROOT/pages/developers/metadata-keys.adoc b/docs/modules/ROOT/pages/developers/metadata-keys.adoc
index 7d5deeb4f7..71ca570b70 100644
--- a/docs/modules/ROOT/pages/developers/metadata-keys.adoc
+++ b/docs/modules/ROOT/pages/developers/metadata-keys.adoc
@@ -16,7 +16,7 @@
= Adding a Metadata Key
Every metadata key Tika can emit is a `Property` constant (or, for runtime-minted names like scraped
-HTML `<meta>` tags, a `PassthroughPrefix`) — there are no bare `String` keys. That closed/open key
+HTML `<meta>` tags, a `KeyPrefix`) — there are no bare `String` keys. That closed/open key
space is tracked in a generated, build-gated registry, so adding a key involves one extra step beyond
writing the Java.
@@ -40,10 +40,10 @@ Naming conventions (frozen for 4.0):
The registry — three JSON files under `tika-metadata-schema/src/main/resources/`, listing every
declared key, every open-namespace prefix, and a field-provenance table — is generated from the live
-`Property`/`PassthroughPrefix` declarations, never hand-edited. A committed copy is the reviewable
+`Property`/`KeyPrefix` declarations, never hand-edited. A committed copy is the reviewable
audit trail (a rename or dropped key shows up as a diff), and CI fails if it's stale.
-Run this after adding, renaming, or removing a `Property` or `PassthroughPrefix`:
+Run this after adding, renaming, or removing a `Property` or `KeyPrefix`:
[source,bash]
----
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/DWG.java b/tika-core/src/main/java/org/apache/tika/metadata/DWG.java
index 67a51a8732..a2b70918ff 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/DWG.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/DWG.java
@@ -23,8 +23,8 @@ public interface DWG {
String DWG_PREFIX = "dwg" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
- PassthroughPrefix RAW_FIELD =
- PassthroughPrefix.file(DWG_PREFIX, "DWGRead JSON header/summary field names");
+ KeyPrefix RAW_FIELD =
+ KeyPrefix.file(DWG_PREFIX, "DWGRead JSON header/summary field names");
Property APPLICATION_NAME = Property.externalText(DWG_PREFIX + "application-name");
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/HTML.java b/tika-core/src/main/java/org/apache/tika/metadata/HTML.java
index a26c979479..97c45da5e3 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/HTML.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/HTML.java
@@ -19,7 +19,7 @@ package org.apache.tika.metadata;
public interface HTML {
String PREFIX_HTML_META = "html" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
- PassthroughPrefix SCRAPED_META = PassthroughPrefix.file(PREFIX_HTML_META,
+ KeyPrefix SCRAPED_META = KeyPrefix.file(PREFIX_HTML_META,
"scraped <meta>/http-equiv/OpenGraph names not mapped to a Property");
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/KeyPrefix.java b/tika-core/src/main/java/org/apache/tika/metadata/KeyPrefix.java
new file mode 100644
index 0000000000..bf244164c6
--- /dev/null
+++ b/tika-core/src/main/java/org/apache/tika/metadata/KeyPrefix.java
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.metadata;
+
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.apache.tika.metadata.Property.PropertyType;
+import org.apache.tika.metadata.Property.ValueType;
+
+/**
+ * A Tika-owned prefix under which keys are passed through from the source: the prefix is fixed, but
+ * each key name comes verbatim from the document or tool, so keys are unbounded and can't be
+ * {@link Property} constants. Declaring one self-registers it, so the open set is enumerable (as
+ * {@link Property} makes the closed set) and lintable: a String write is legitimate iff its key is a
+ * registered {@link Property} or its prefix is a registered {@code KeyPrefix}.
+ *
+ * <p><strong>{@code KeyPrefix} instances are declaration-time constants.</strong> Declare one as a
+ * {@code static final} field, the same way a curated {@link Property} constant is declared — never
+ * construct one from document-derived text, and never construct one per-parse: the registry is
+ * static and unbounded per-parse construction grows it forever (and now also throws, see below).
+ *
+ * @since Apache Tika 4.0.0
+ */
+public final class KeyPrefix {
+
+ public enum Provenance { FILE, TOOL }
+
+ /** Delimiters accepted at the end of a prefix; kept intentionally loose (some declarations
+ * predate a single convention) — see {@code ReservedNamespaces} for the reserved-name check. */
+ private static final char[] TRAILING_DELIMITERS = {':', '.', '-', '_'};
+
+ private static final Map<String, KeyPrefix> REGISTRY = new ConcurrentHashMap<>();
+
+ private final String prefix;
+ private final Provenance provenance;
+ private final String description;
+
+ private KeyPrefix(String prefix, Provenance provenance, String description) {
+ if (prefix == null || prefix.isEmpty()) {
+ throw new IllegalArgumentException("KeyPrefix prefix must not be null or empty");
+ }
+ if (ReservedNamespaces.isTikaNative(prefix)) {
+ throw new IllegalArgumentException("'" + prefix + "' is in the reserved Tika-native "
+ + "namespace (tk:/X-TIKA:); it cannot be a KeyPrefix");
+ }
+ if (!hasTrailingDelimiter(prefix)) {
+ throw new IllegalArgumentException("'" + prefix + "' must end with one of "
+ + Arrays.toString(TRAILING_DELIMITERS));
+ }
+ this.prefix = prefix;
+ this.provenance = provenance;
+ this.description = description;
+ KeyPrefix incumbent = REGISTRY.putIfAbsent(prefix, this);
+ if (incumbent != null) {
+ throw new IllegalStateException(
+ "KeyPrefix '" + prefix + "' is already registered; KeyPrefix instances are "
+ + "declaration-time constants and must be constructed exactly once "
+ + "(never from document-derived text, never per-parse)");
+ }
+ }
+
+ private static boolean hasTrailingDelimiter(String prefix) {
+ char last = prefix.charAt(prefix.length() - 1);
+ for (char d : TRAILING_DELIMITERS) {
+ if (d == last) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ public static KeyPrefix file(String prefix, String description) {
+ return new KeyPrefix(prefix, Provenance.FILE, description);
+ }
+
+ public static KeyPrefix tool(String prefix, String description) {
+ return new KeyPrefix(prefix, Provenance.TOOL, description);
+ }
+
+ /** The full key for a source-derived {@code suffix}. */
+ public String key(String suffix) {
+ return prefix + suffix;
+ }
+
+ public String prefix() {
+ return prefix;
+ }
+
+ public Provenance provenance() {
+ return provenance;
+ }
+
+ public String description() {
+ return description;
+ }
+
+ /** Declared prefixes, from loaded classes only. */
+ public static Collection<KeyPrefix> registered() {
+ return Collections.unmodifiableCollection(REGISTRY.values());
+ }
+
+ // ---- Typed minting factories -----------------------------------------------------
+ // Each mints an UNREGISTERED Property (Property.mintUnregistered) named prefix + name,
+ // shaped like the corresponding external* Property factory. Mint per call for doc-derived
+ // names: minted Properties are cheap, unregistered value objects, GC'd with the parse.
+ // A bounded, known vocabulary belongs in a curated Property constant instead (that IS the
+ // cache, with registry/CI coverage) — never build a name-keyed static cache here, that
+ // recreates the registry-growth leak this design avoids.
+
+ private static void requireNonEmptyName(String name) {
+ if (name == null || name.isEmpty()) {
+ throw new IllegalArgumentException("KeyPrefix-minted Property name must not be null "
+ + "or empty");
+ }
+ }
+
+ /** Mints an unregistered SIMPLE/TEXT Property, like {@link Property#externalText}. */
+ public Property text(String name) {
+ requireNonEmptyName(name);
+ return Property.mintUnregistered(key(name), false, PropertyType.SIMPLE, ValueType.TEXT, null);
+ }
+
+ /** Mints an unregistered BAG/TEXT Property, like {@link Property#externalTextBag}. */
+ public Property textBag(String name) {
+ requireNonEmptyName(name);
+ return Property.mintUnregistered(key(name), false, PropertyType.BAG, ValueType.TEXT, null);
+ }
+
+ /** Mints an unregistered SIMPLE/DATE Property, like {@link Property#externalDate}. */
+ public Property date(String name) {
+ requireNonEmptyName(name);
+ return Property.mintUnregistered(key(name), false, PropertyType.SIMPLE, ValueType.DATE, null);
+ }
+
+ /** Mints an unregistered SIMPLE/INTEGER Property, like {@link Property#externalInteger}. */
+ public Property integer(String name) {
+ requireNonEmptyName(name);
+ return Property.mintUnregistered(key(name), false, PropertyType.SIMPLE, ValueType.INTEGER,
+ null);
+ }
+
+ /** Mints an unregistered SIMPLE/REAL Property, like {@link Property#externalReal}. */
+ public Property real(String name) {
+ requireNonEmptyName(name);
+ return Property.mintUnregistered(key(name), false, PropertyType.SIMPLE, ValueType.REAL, null);
+ }
+
+ /** Mints an unregistered SIMPLE/BOOLEAN Property, like {@link Property#externalBoolean}. */
+ public Property bool(String name) {
+ requireNonEmptyName(name);
+ return Property.mintUnregistered(key(name), false, PropertyType.SIMPLE, ValueType.BOOLEAN,
+ null);
+ }
+}
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/MAPI.java b/tika-core/src/main/java/org/apache/tika/metadata/MAPI.java
index 74657ce85b..5b9b940913 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/MAPI.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/MAPI.java
@@ -27,7 +27,7 @@ public interface MAPI {
String PREFIX_MAPI_META = "mapi" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
String PREFIX_MAPI_ATTACH_META = "mapi:attach" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
String PREFIX_MAPI_PROPERTY = PREFIX_MAPI_META + "property" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
- PassthroughPrefix PROPERTY = PassthroughPrefix.file(PREFIX_MAPI_PROPERTY, "MAPI named properties");
+ KeyPrefix PROPERTY = KeyPrefix.file(PREFIX_MAPI_PROPERTY, "MAPI named properties");
/**
* MAPI message class. What type of .msg/MAPI file is it?
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/Message.java b/tika-core/src/main/java/org/apache/tika/metadata/Message.java
index cf711eeec8..e069a396a6 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/Message.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/Message.java
@@ -29,7 +29,7 @@ public interface Message {
String MESSAGE_RAW_HEADER_PREFIX =
MESSAGE_PREFIX + "raw-header" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
- PassthroughPrefix RAW_HEADER = PassthroughPrefix.file(MESSAGE_RAW_HEADER_PREFIX,
+ KeyPrefix RAW_HEADER = KeyPrefix.file(MESSAGE_RAW_HEADER_PREFIX,
"RFC822 / Outlook raw email header names");
Property MESSAGE_RECIPIENT_ADDRESS =
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/Office.java b/tika-core/src/main/java/org/apache/tika/metadata/Office.java
index 2a348f45d9..f2d7ccc686 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/Office.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/Office.java
@@ -37,7 +37,7 @@ public interface Office {
*/
String USER_DEFINED_METADATA_NAME_PREFIX = "custom:";
- PassthroughPrefix USER_DEFINED = PassthroughPrefix.file(USER_DEFINED_METADATA_NAME_PREFIX,
+ KeyPrefix USER_DEFINED = KeyPrefix.file(USER_DEFINED_METADATA_NAME_PREFIX,
"OOXML/OLE2/ODF user-defined document properties");
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/PDF.java b/tika-core/src/main/java/org/apache/tika/metadata/PDF.java
index c7bda2fdf4..08f2645208 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/PDF.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/PDF.java
@@ -51,8 +51,8 @@ public interface PDF {
String PDF_DOC_INFO_CUSTOM_PREFIX =
PDF_DOC_INFO_PREFIX + "custom" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
- PassthroughPrefix DOC_INFO_CUSTOM =
- PassthroughPrefix.file(PDF_DOC_INFO_CUSTOM_PREFIX, "PDF Info-dict custom keys");
+ KeyPrefix DOC_INFO_CUSTOM =
+ KeyPrefix.file(PDF_DOC_INFO_CUSTOM_PREFIX, "PDF Info-dict custom keys");
Property DOC_INFO_CREATED = Property.internalDate(PDF_DOC_INFO_PREFIX + "created");
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/PassthroughPrefix.java b/tika-core/src/main/java/org/apache/tika/metadata/PassthroughPrefix.java
deleted file mode 100644
index ea17d3b555..0000000000
--- a/tika-core/src/main/java/org/apache/tika/metadata/PassthroughPrefix.java
+++ /dev/null
@@ -1,79 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.tika.metadata;
-
-import java.util.Collection;
-import java.util.Collections;
-import java.util.Map;
-import java.util.concurrent.ConcurrentHashMap;
-
-/**
- * A Tika-owned prefix under which keys are passed through from the source: the prefix is fixed, but
- * each key name comes verbatim from the document or tool, so keys are unbounded and can't be
- * {@link Property} constants. Declaring one self-registers it, so the open set is enumerable (as
- * {@link Property} makes the closed set) and lintable: a String write is legitimate iff its key is a
- * registered {@link Property} or its prefix is a registered {@code PassthroughPrefix}.
- *
- * @since Apache Tika 4.0.0
- */
-public final class PassthroughPrefix {
-
- public enum Provenance { FILE, TOOL }
-
- private static final Map<String, PassthroughPrefix> REGISTRY = new ConcurrentHashMap<>();
-
- private final String prefix;
- private final Provenance provenance;
- private final String description;
-
- private PassthroughPrefix(String prefix, Provenance provenance, String description) {
- this.prefix = prefix;
- this.provenance = provenance;
- this.description = description;
- REGISTRY.put(prefix, this);
- }
-
- public static PassthroughPrefix file(String prefix, String description) {
- return new PassthroughPrefix(prefix, Provenance.FILE, description);
- }
-
- public static PassthroughPrefix tool(String prefix, String description) {
- return new PassthroughPrefix(prefix, Provenance.TOOL, description);
- }
-
- /** The full key for a source-derived {@code suffix}. */
- public String key(String suffix) {
- return prefix + suffix;
- }
-
- public String prefix() {
- return prefix;
- }
-
- public Provenance provenance() {
- return provenance;
- }
-
- public String description() {
- return description;
- }
-
- /** Declared prefixes, from loaded classes only. */
- public static Collection<PassthroughPrefix> registered() {
- return Collections.unmodifiableCollection(REGISTRY.values());
- }
-}
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/RTFMetadata.java b/tika-core/src/main/java/org/apache/tika/metadata/RTFMetadata.java
index 94aefbc85b..f1eb58d213 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/RTFMetadata.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/RTFMetadata.java
@@ -26,7 +26,7 @@ public interface RTFMetadata {
String RTF_PICT_META_PREFIX = "rtf:pict:";
/** Open (file-controlled) RTF embedded-object property names built off {@link #RTF_PICT_META_PREFIX}. */
- PassthroughPrefix PICT = PassthroughPrefix.file(RTF_PICT_META_PREFIX,
+ KeyPrefix PICT = KeyPrefix.file(RTF_PICT_META_PREFIX,
"RTF embedded picture-object property pairs (sn/sv); file-controlled names");
/**
diff --git a/tika-core/src/test/java/org/apache/tika/metadata/KeyPrefixTest.java b/tika-core/src/test/java/org/apache/tika/metadata/KeyPrefixTest.java
new file mode 100644
index 0000000000..25bd8e29d9
--- /dev/null
+++ b/tika-core/src/test/java/org/apache/tika/metadata/KeyPrefixTest.java
@@ -0,0 +1,182 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.metadata;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import org.junit.jupiter.api.Test;
+
+import org.apache.tika.metadata.Property.PropertyType;
+import org.apache.tika.metadata.Property.ValueType;
+
+/**
+ * Stage 3: the {@code PassthroughPrefix} -> {@code KeyPrefix} rename, constructor validation,
+ * and the typed unregistered-Property minting factories.
+ */
+public class KeyPrefixTest {
+
+ // Every registered prefix is a unique, unrepeatable static registration -- give each test
+ // its own prefix (nanoTime-suffixed, following PropertyReservedNameTest's convention) so
+ // tests never collide with each other or with real parser-declared prefixes.
+ private static String uniquePrefix(String label) {
+ return "keyprefix-test-" + label + "-" + System.nanoTime() + ":";
+ }
+
+ @Test
+ public void testRejectsNullPrefix() {
+ assertThrows(IllegalArgumentException.class, () -> KeyPrefix.file(null, "d"));
+ }
+
+ @Test
+ public void testRejectsEmptyPrefix() {
+ assertThrows(IllegalArgumentException.class, () -> KeyPrefix.file("", "d"));
+ }
+
+ @Test
+ public void testRejectsReservedTkPrefix() {
+ assertThrows(IllegalArgumentException.class, () -> KeyPrefix.file("tk:", "d"));
+ }
+
+ @Test
+ public void testRejectsReservedLegacyXTikaPrefix() {
+ assertThrows(IllegalArgumentException.class, () -> KeyPrefix.file("X-TIKA:", "d"));
+ }
+
+ @Test
+ public void testRejectsMissingTrailingDelimiter() {
+ String prefix = "keyprefix-test-no-delim-" + System.nanoTime();
+ assertThrows(IllegalArgumentException.class, () -> KeyPrefix.file(prefix, "d"));
+ }
+
+ @Test
+ public void testAcceptsAllCurrentlyUsedTrailingDelimiters() {
+ // ':' '.' '-' '_' -- the population enumerated pre-implementation (netcdf: envi.
+ // grobid:header_ MboxParser- etc). A later stage normalizes these to ':'; stage 3 must
+ // not break the still-live population.
+ for (char delim : new char[]{':', '.', '-', '_'}) {
+ String prefix = uniquePrefix("delim-" + delim) + delim;
+ KeyPrefix.file(prefix, "d"); // must not throw
+ }
+ }
+
+ @Test
+ public void testDuplicateRegistrationThrows() {
+ String prefix = uniquePrefix("dup");
+ KeyPrefix.file(prefix, "first");
+ assertThrows(IllegalStateException.class, () -> KeyPrefix.tool(prefix, "second"));
+ }
+
+ @Test
+ public void testKeyUnchanged() {
+ KeyPrefix kp = KeyPrefix.file(uniquePrefix("key"), "d");
+ assertEquals(kp.prefix() + "suffix", kp.key("suffix"));
+ }
+
+ @Test
+ public void testFileProvenancePreserved() {
+ KeyPrefix kp = KeyPrefix.file(uniquePrefix("prov-file"), "d");
+ assertEquals(KeyPrefix.Provenance.FILE, kp.provenance());
+ }
+
+ @Test
+ public void testToolProvenancePreserved() {
+ KeyPrefix kp = KeyPrefix.tool(uniquePrefix("prov-tool"), "d");
+ assertEquals(KeyPrefix.Provenance.TOOL, kp.provenance());
+ }
+
+ @Test
+ public void testTextFactoryShape() {
+ KeyPrefix kp = KeyPrefix.file(uniquePrefix("text"), "d");
+ Property p = kp.text("name");
+ assertEquals(kp.prefix() + "name", p.getName());
+ assertEquals(PropertyType.SIMPLE, p.getPropertyType());
+ assertEquals(ValueType.TEXT, p.getValueType());
+ assertNull(Property.get(p.getName()), "minted Property must not register");
+ }
+
+ @Test
+ public void testTextBagFactoryShape() {
+ KeyPrefix kp = KeyPrefix.file(uniquePrefix("textbag"), "d");
+ Property p = kp.textBag("name");
+ assertEquals(kp.prefix() + "name", p.getName());
+ assertEquals(PropertyType.BAG, p.getPropertyType());
+ assertEquals(ValueType.TEXT, p.getValueType());
+ assertNull(Property.get(p.getName()));
+ }
+
+ @Test
+ public void testDateFactoryShape() {
+ KeyPrefix kp = KeyPrefix.file(uniquePrefix("date"), "d");
+ Property p = kp.date("name");
+ assertEquals(kp.prefix() + "name", p.getName());
+ assertEquals(PropertyType.SIMPLE, p.getPropertyType());
+ assertEquals(ValueType.DATE, p.getValueType());
+ assertNull(Property.get(p.getName()));
+ }
+
+ @Test
+ public void testIntegerFactoryShape() {
+ KeyPrefix kp = KeyPrefix.file(uniquePrefix("integer"), "d");
+ Property p = kp.integer("name");
+ assertEquals(kp.prefix() + "name", p.getName());
+ assertEquals(PropertyType.SIMPLE, p.getPropertyType());
+ assertEquals(ValueType.INTEGER, p.getValueType());
+ assertNull(Property.get(p.getName()));
+ }
+
+ @Test
+ public void testRealFactoryShape() {
+ KeyPrefix kp = KeyPrefix.file(uniquePrefix("real"), "d");
+ Property p = kp.real("name");
+ assertEquals(kp.prefix() + "name", p.getName());
+ assertEquals(PropertyType.SIMPLE, p.getPropertyType());
+ assertEquals(ValueType.REAL, p.getValueType());
+ assertNull(Property.get(p.getName()));
+ }
+
+ @Test
+ public void testBoolFactoryShape() {
+ KeyPrefix kp = KeyPrefix.file(uniquePrefix("bool"), "d");
+ Property p = kp.bool("name");
+ assertEquals(kp.prefix() + "name", p.getName());
+ assertEquals(PropertyType.SIMPLE, p.getPropertyType());
+ assertEquals(ValueType.BOOLEAN, p.getValueType());
+ assertNull(Property.get(p.getName()));
+ }
+
+ @Test
+ public void testFactoriesRejectEmptyName() {
+ KeyPrefix kp = KeyPrefix.file(uniquePrefix("empty-name"), "d");
+ assertThrows(IllegalArgumentException.class, () -> kp.text(""));
+ assertThrows(IllegalArgumentException.class, () -> kp.text(null));
+ assertThrows(IllegalArgumentException.class, () -> kp.textBag(""));
+ assertThrows(IllegalArgumentException.class, () -> kp.date(""));
+ assertThrows(IllegalArgumentException.class, () -> kp.integer(""));
+ assertThrows(IllegalArgumentException.class, () -> kp.real(""));
+ assertThrows(IllegalArgumentException.class, () -> kp.bool(""));
+ }
+
+ @Test
+ public void testRegisteredIncludesDeclaredPrefix() {
+ String prefix = uniquePrefix("registered");
+ KeyPrefix kp = KeyPrefix.file(prefix, "d");
+ assertTrue(KeyPrefix.registered().contains(kp));
+ }
+}
diff --git a/tika-metadata-schema/README.md b/tika-metadata-schema/README.md
index 94968e36fa..7650ed36ca 100644
--- a/tika-metadata-schema/README.md
+++ b/tika-metadata-schema/README.md
@@ -36,7 +36,7 @@ declare a `Property` field, force-loads them, reads the global `Property` table,
sorted JSON. `MetadataSchemaTest` regenerates in-memory and asserts it matches the committed file, so
the registry can never drift from the declarations.
-Regenerate after adding/changing a `Property` **or** a `PassthroughPrefix` (writes all three files):
+Regenerate after adding/changing a `Property` **or** a `KeyPrefix` (writes all three files):
```
tika-metadata-schema/regen.sh
```
@@ -50,8 +50,8 @@ The **prefixes** under which parsers mint file-controlled key names at runtime
`html:`, OOXML `custom:`, email `Message:Raw-Header:`, Access `MDB_PROP:`, Vorbis comments, FLV
attributes, unmapped image/XMP tags, …). Each record: `{ prefix, provenance, description }`.
-**Generated from the `PassthroughPrefix` declarations, never hand-edited.** Every such prefix is a
-registered `PassthroughPrefix` constant; `SchemaGenerator` reads that registry the same way it reads
+**Generated from the `KeyPrefix` declarations, never hand-edited.** Every such prefix is a
+registered `KeyPrefix` constant; `SchemaGenerator` reads that registry the same way it reads
the `Property` table, and `MetadataSchemaTest` gates it identically. Adding a passthrough prefix in a
parser and forgetting to regenerate fails the build.
diff --git a/tika-metadata-schema/regen.sh b/tika-metadata-schema/regen.sh
index 4d20fa7f45..1874468e57 100755
--- a/tika-metadata-schema/regen.sh
+++ b/tika-metadata-schema/regen.sh
@@ -2,7 +2,7 @@
#
# Regenerates and validates the metadata key registry (tika-metadata-schema).
#
-# Run this after adding, renaming, or removing a Property or PassthroughPrefix
+# Run this after adding, renaming, or removing a Property or KeyPrefix
# constant anywhere in tika-core or the standard parser bundle. It replaces the
# multi-step manual sequence in .skills/metadata-schema.md with one command:
# install the dependency modules, regenerate the three registry files, sanity
@@ -12,7 +12,7 @@
# tika-metadata-schema/regen.sh [--skip-install] [--skip-tests]
#
# --skip-install skip the -am install step (only safe if no Property/
-# PassthroughPrefix classes outside tika-metadata-schema
+# KeyPrefix classes outside tika-metadata-schema
# changed since the last install)
# --skip-tests skip the final gate-test run, for a faster inner loop
#
@@ -53,7 +53,7 @@ REGISTRY_FILES=(
if [ "$SKIP_INSTALL" -eq 0 ]; then
echo "==> Installing tika-metadata-schema + its dependency modules (tika-core, standard parsers)"
- echo " so newly added Property/PassthroughPrefix classes are on the scan classpath."
+ echo " so newly added Property/KeyPrefix classes are on the scan classpath."
echo " (skip with --skip-install if you already did this)"
./mvnw -Pfast -DskipTests -pl tika-metadata-schema -am install "$MVN_REPO_OPT"
fi
@@ -92,4 +92,4 @@ if [ "$SKIP_TESTS" -eq 0 ]; then
./mvnw -pl tika-metadata-schema test "$MVN_REPO_OPT"
fi
-echo "==> Done. Review the diff above, then commit the Property/PassthroughPrefix change and the regenerated JSON together."
+echo "==> Done. Review the diff above, then commit the Property/KeyPrefix change and the regenerated JSON together."
diff --git a/tika-metadata-schema/src/main/java/org/apache/tika/metadata/schema/SchemaGenerator.java b/tika-metadata-schema/src/main/java/org/apache/tika/metadata/schema/SchemaGenerator.java
index 8a552319cc..2be82a94b4 100644
--- a/tika-metadata-schema/src/main/java/org/apache/tika/metadata/schema/SchemaGenerator.java
+++ b/tika-metadata-schema/src/main/java/org/apache/tika/metadata/schema/SchemaGenerator.java
@@ -32,7 +32,7 @@ import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import org.apache.tika.digest.DigestDef;
-import org.apache.tika.metadata.PassthroughPrefix;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Property;
/**
@@ -54,8 +54,8 @@ public final class SchemaGenerator {
// Field/parameter descriptors in a .class constant pool: a class referencing one is force-loaded.
private static final byte[] PROP_DESC =
"Lorg/apache/tika/metadata/Property;".getBytes(StandardCharsets.ISO_8859_1);
- private static final byte[] PASSTHROUGH_DESC =
- "Lorg/apache/tika/metadata/PassthroughPrefix;".getBytes(StandardCharsets.ISO_8859_1);
+ private static final byte[] KEY_PREFIX_DESC =
+ "Lorg/apache/tika/metadata/KeyPrefix;".getBytes(StandardCharsets.ISO_8859_1);
private SchemaGenerator() {
}
@@ -118,7 +118,7 @@ public final class SchemaGenerator {
return fieldTableJson(rows);
}
- /** Scans the classpath, force-loads every Property/PassthroughPrefix-bearing class (static init
+ /** Scans the classpath, force-loads every Property/KeyPrefix-bearing class (static init
* registers the constants), and returns the loaded class names. */
private static List<String> scanClasspath(ClassLoader cl) throws IOException {
List<String> loaded = new ArrayList<>();
@@ -160,7 +160,7 @@ public final class SchemaGenerator {
private static void maybeLoad(String classPath, byte[] bytes, ClassLoader cl, List<String> loaded) {
if (!classPath.startsWith("org/apache/tika/")
- || (!contains(bytes, PROP_DESC) && !contains(bytes, PASSTHROUGH_DESC))) {
+ || (!contains(bytes, PROP_DESC) && !contains(bytes, KEY_PREFIX_DESC))) {
return;
}
String cn = classPath.substring(0, classPath.length() - 6).replace('/', '.');
@@ -222,7 +222,7 @@ public final class SchemaGenerator {
/** Declared passthrough prefixes as stable JSON. Call after {@link #generate()} has loaded classes. */
public static String passthroughJson() {
TreeMap<String, String[]> m = new TreeMap<>();
- for (PassthroughPrefix p : PassthroughPrefix.registered()) {
+ for (KeyPrefix p : KeyPrefix.registered()) {
m.put(p.prefix(), new String[]{p.provenance().name(), p.description()});
}
StringBuilder sb = new StringBuilder("[\n");
diff --git a/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataCoverageTest.java b/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataCoverageTest.java
index 4980d770d1..eedd98975b 100644
--- a/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataCoverageTest.java
+++ b/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataCoverageTest.java
@@ -34,7 +34,7 @@ import org.junit.jupiter.api.Test;
* Completeness backstop for the standard-scoped registry. {@link MetadataSchemaTest} proves the
* registry matches the declarations it <em>scans</em>; this proves nothing <em>escapes</em> the
* scan. It greps the source tree for every module declaring a {@code Property} or
- * {@code PassthroughPrefix} and fails if one is neither on the registry classpath nor an explicit
+ * {@code KeyPrefix} and fails if one is neither on the registry classpath nor an explicit
* {@link #OUT_OF_SCOPE} family — so a new key-bearing parser cannot slip in silently.
*
* <p>Reads source (always present) for the declaring set and the runtime classpath for the scanned
@@ -53,7 +53,7 @@ public class MetadataCoverageTest {
// Source markers for an actual key declaration (a factory call), not a mere type reference.
private static final String[] DECL_MARKERS = {
"Property.internal", "Property.external", "Property.composite",
- "PassthroughPrefix.file(", "PassthroughPrefix.tool("};
+ "KeyPrefix.file(", "KeyPrefix.tool("};
private static final String SRC_MAIN = sep("src", "main") + File.separator + "java";
private static final String SRC_TEST = sep("src", "test");
diff --git a/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataSchemaTest.java b/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataSchemaTest.java
index 5530b82b10..25cf6d8aef 100644
--- a/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataSchemaTest.java
+++ b/tika-metadata-schema/src/test/java/org/apache/tika/metadata/schema/MetadataSchemaTest.java
@@ -26,7 +26,7 @@ import org.junit.jupiter.api.Test;
/**
* The CI gate: regenerate both registries from the live {@link org.apache.tika.metadata.Property}
- * and {@link org.apache.tika.metadata.PassthroughPrefix} declarations and assert each matches its
+ * and {@link org.apache.tika.metadata.KeyPrefix} declarations and assert each matches its
* committed file. If a declaration is added/changed/renamed without regenerating, this fails — the
* registries cannot silently drift.
*/
@@ -44,7 +44,7 @@ public class MetadataSchemaTest {
@Test
public void committedOpenNamespacesMatchDeclarations() throws Exception {
- // generate() first, so the classpath scan force-loads the PassthroughPrefix declarations.
+ // generate() first, so the classpath scan force-loads the KeyPrefix declarations.
SchemaGenerator.generate();
assertEquals(committed(OPEN), SchemaGenerator.passthroughJson(),
"metadata-open-namespaces.json is stale. Run tika-metadata-schema/regen.sh and "
diff --git a/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/envi/EnviHeaderParser.java b/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/envi/EnviHeaderParser.java
index a4d902c882..19ef1f1cc4 100644
--- a/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/envi/EnviHeaderParser.java
+++ b/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/envi/EnviHeaderParser.java
@@ -33,8 +33,8 @@ import org.apache.tika.detect.AutoDetectReader;
import org.apache.tika.detect.EncodingDetector;
import org.apache.tika.exception.TikaException;
import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.AbstractEncodingDetectorParser;
import org.apache.tika.parser.ParseContext;
@@ -44,8 +44,8 @@ import org.apache.tika.sax.XHTMLContentHandler;
public class EnviHeaderParser extends AbstractEncodingDetectorParser {
public static final String ENVI_MIME_TYPE = "application/envi.hdr";
- public static final PassthroughPrefix ENVI =
- PassthroughPrefix.file("envi.", "ENVI header field names");
+ public static final KeyPrefix ENVI =
+ KeyPrefix.file("envi.", "ENVI header field names");
private static final long serialVersionUID = -1479368523072408091L;
private static final Logger LOG = LoggerFactory.getLogger(EnviHeaderParser.class);
private static final Set<MediaType> SUPPORTED_TYPES =
diff --git a/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/grib/GribParser.java b/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/grib/GribParser.java
index 073eb7bf25..94f819f9f2 100644
--- a/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/grib/GribParser.java
+++ b/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/grib/GribParser.java
@@ -36,8 +36,8 @@ import org.apache.tika.annotation.TikaComponent;
import org.apache.tika.exception.TikaException;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.ClimateForcast;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
@@ -48,8 +48,8 @@ import org.apache.tika.sax.XHTMLContentHandler;
public class GribParser implements Parser {
public static final String GRIB_MIME_TYPE = "application/x-grib2";
- public static final PassthroughPrefix GRIB =
- PassthroughPrefix.file("grib:", "GRIB global attribute names");
+ public static final KeyPrefix GRIB =
+ KeyPrefix.file("grib:", "GRIB global attribute names");
private static final long serialVersionUID = 7855458954474247655L;
private final Set<MediaType> SUPPORTED_TYPES =
Collections.singleton(MediaType.application("x-grib2"));
diff --git a/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/netcdf/NetCDFParser.java b/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/netcdf/NetCDFParser.java
index fe2300a976..1cc2cef3e2 100644
--- a/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/netcdf/NetCDFParser.java
+++ b/tika-parsers/tika-parsers-extended/tika-parser-scientific-module/src/main/java/org/apache/tika/parser/netcdf/NetCDFParser.java
@@ -34,8 +34,8 @@ import org.apache.tika.annotation.TikaComponent;
import org.apache.tika.exception.TikaException;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.ClimateForcast;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
@@ -57,8 +57,8 @@ public class NetCDFParser implements Parser {
*/
private static final long serialVersionUID = -5940938274907708665L;
- public static final PassthroughPrefix NETCDF =
- PassthroughPrefix.file("netcdf:", "NetCDF global attribute names");
+ public static final KeyPrefix NETCDF =
+ KeyPrefix.file("netcdf:", "NetCDF global attribute names");
private final Set<MediaType> SUPPORTED_TYPES =
Collections.singleton(MediaType.application("x-netcdf"));
diff --git a/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/ctakes/CTAKESContentHandler.java b/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/ctakes/CTAKESContentHandler.java
index bb6ae8cdcf..9decddcad4 100644
--- a/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/ctakes/CTAKESContentHandler.java
+++ b/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/ctakes/CTAKESContentHandler.java
@@ -26,8 +26,8 @@ import org.xml.sax.ContentHandler;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.sax.ContentHandlerDecorator;
/**
@@ -42,8 +42,8 @@ import org.apache.tika.sax.ContentHandlerDecorator;
public class CTAKESContentHandler extends ContentHandlerDecorator {
// Prefix used for metadata including cTAKES annotations
public static String CTAKES_META_PREFIX = "ctakes:";
- public static final PassthroughPrefix CTAKES =
- PassthroughPrefix.tool(CTAKES_META_PREFIX, "cTAKES annotation type names");
+ public static final KeyPrefix CTAKES =
+ KeyPrefix.tool(CTAKES_META_PREFIX, "cTAKES annotation type names");
// Configuration object for CTAKESContentHandler
private CTAKESConfig config = null;
diff --git a/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/journal/GrobidRESTParser.java b/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/journal/GrobidRESTParser.java
index 3eab5ee4de..1ad472cf1c 100644
--- a/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/journal/GrobidRESTParser.java
+++ b/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/journal/GrobidRESTParser.java
@@ -34,8 +34,8 @@ import org.xml.sax.ContentHandler;
import org.apache.tika.annotation.TikaComponent;
import org.apache.tika.exception.TikaException;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.parser.ParseContext;
/**
@@ -47,8 +47,8 @@ public class GrobidRESTParser {
private static final Logger LOG = LoggerFactory.getLogger(GrobidRESTParser.class);
- public static final PassthroughPrefix GROBID_HEADER =
- PassthroughPrefix.tool("grobid:header_", "GROBID-inferred header fields");
+ public static final KeyPrefix GROBID_HEADER =
+ KeyPrefix.tool("grobid:header_", "GROBID-inferred header fields");
private static final String GROBID_REST_HOST = "http://localhost:8070";
private static final String GROBID_ISALIVE_PATH = "/api/isalive";
diff --git a/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/ner/NamedEntityParser.java b/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/ner/NamedEntityParser.java
index 52df871ded..9cd2f8a0d5 100644
--- a/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/ner/NamedEntityParser.java
+++ b/tika-parsers/tika-parsers-ml/tika-parser-nlp-module/src/main/java/org/apache/tika/parser/ner/NamedEntityParser.java
@@ -37,8 +37,8 @@ import org.apache.tika.annotation.TikaComponent;
import org.apache.tika.config.loader.TikaLoader;
import org.apache.tika.exception.TikaException;
import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
@@ -62,8 +62,8 @@ public class NamedEntityParser implements Parser {
public static final Logger LOG = LoggerFactory.getLogger(NamedEntityParser.class);
public static final Set<MediaType> MEDIA_TYPES = new HashSet<>();
public static final String MD_KEY_PREFIX = "NER_";
- public static final PassthroughPrefix NER =
- PassthroughPrefix.tool(MD_KEY_PREFIX, "named-entity types");
+ public static final KeyPrefix NER =
+ KeyPrefix.tool(MD_KEY_PREFIX, "named-entity types");
public static final String DEFAULT_NER_IMPL =
OpenNLPNERecogniser.class.getName() + "," + RegexNERecogniser.class.getName();
public static final String SYS_PROP_NER_IMPL = "ner.impl.class";
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-apple-module/src/main/java/org/apache/tika/parser/iwork/NumbersContentHandler.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-apple-module/src/main/java/org/apache/tika/parser/iwork/NumbersContentHandler.java
index 87237ffca9..cb292057f1 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-apple-module/src/main/java/org/apache/tika/parser/iwork/NumbersContentHandler.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-apple-module/src/main/java/org/apache/tika/parser/iwork/NumbersContentHandler.java
@@ -23,16 +23,16 @@ import org.xml.sax.Attributes;
import org.xml.sax.SAXException;
import org.xml.sax.helpers.DefaultHandler;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.Office;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.sax.XHTMLContentHandler;
class NumbersContentHandler extends DefaultHandler {
- private static final PassthroughPrefix NUMBERS_METADATA =
- PassthroughPrefix.file("numbers:", "iWork Numbers document metadata element names");
+ private static final KeyPrefix NUMBERS_METADATA =
+ KeyPrefix.file("numbers:", "iWork Numbers document metadata element names");
private final XHTMLContentHandler xhtml;
private final Metadata metadata;
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-apple-module/src/main/java/org/apache/tika/parser/iwork/iwana/IWork13PackageParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-apple-module/src/main/java/org/apache/tika/parser/iwork/iwana/IWork13PackageParser.java
index 1ffb5fa5ec..797157e49f 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-apple-module/src/main/java/org/apache/tika/parser/iwork/iwana/IWork13PackageParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-apple-module/src/main/java/org/apache/tika/parser/iwork/iwana/IWork13PackageParser.java
@@ -44,8 +44,8 @@ import org.apache.tika.extractor.EmbeddedDocumentExtractor;
import org.apache.tika.extractor.EmbeddedDocumentUtil;
import org.apache.tika.io.TemporaryResources;
import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.metadata.Property;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
@@ -67,8 +67,8 @@ public class IWork13PackageParser implements Parser {
Property.externalText(IWORKS_PREFIX + "document-id");
public static final Property IWORKS_BUILD_VERSION_HISTORY =
Property.externalTextBag(IWORKS_PREFIX + "build-version-history");
- public static final PassthroughPrefix IWORKS_PROPERTIES =
- PassthroughPrefix.file(IWORKS_PREFIX, "iWork '13 plist document properties");
+ public static final KeyPrefix IWORKS_PROPERTIES =
+ KeyPrefix.file(IWORKS_PREFIX, "iWork '13 plist document properties");
private final static Set<MediaType> supportedTypes = Collections.unmodifiableSet(
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/ogg/OggAudioParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/ogg/OggAudioParser.java
index 7d0b043889..f583c4fb56 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/ogg/OggAudioParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/ogg/OggAudioParser.java
@@ -41,8 +41,8 @@ import org.apache.tika.extractor.EmbeddedDocumentExtractor;
import org.apache.tika.extractor.EmbeddedDocumentUtil;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Audio;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.metadata.XMP;
import org.apache.tika.metadata.XMPDM;
@@ -59,8 +59,8 @@ import org.apache.tika.sax.XHTMLContentHandler;
public abstract class OggAudioParser extends AbstractParser {
private static final long serialVersionUID = 5168743829615945633L;
- private static final PassthroughPrefix VORBIS =
- PassthroughPrefix.file("vorbis:", "Vorbis comment field names");
+ private static final KeyPrefix VORBIS =
+ KeyPrefix.file("vorbis:", "Vorbis comment field names");
/**
* Comment holding an embedded picture (e.g. cover art) as a base64
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/video/FLVParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/video/FLVParser.java
index 32900f319e..036362338d 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/video/FLVParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-audiovideo-module/src/main/java/org/apache/tika/parser/video/FLVParser.java
@@ -35,8 +35,8 @@ import org.xml.sax.SAXException;
import org.apache.tika.annotation.TikaComponent;
import org.apache.tika.exception.TikaException;
import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
@@ -77,8 +77,8 @@ public class FLVParser implements Parser {
private static int TYPE_METADATA = 0x12;
private static byte MASK_AUDIO = 1;
private static byte MASK_VIDEO = 4;
- private static final PassthroughPrefix FLV =
- PassthroughPrefix.file("flv:", "FLV onMetaData tag keys");
+ private static final KeyPrefix FLV =
+ KeyPrefix.file("flv:", "FLV onMetaData tag keys");
public Set<MediaType> getSupportedTypes(ParseContext context) {
return SUPPORTED_TYPES;
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-cad-module/src/main/java/org/apache/tika/parser/dwg/DWGParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-cad-module/src/main/java/org/apache/tika/parser/dwg/DWGParser.java
index a03578d0eb..4c7eafcd03 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-cad-module/src/main/java/org/apache/tika/parser/dwg/DWGParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-cad-module/src/main/java/org/apache/tika/parser/dwg/DWGParser.java
@@ -31,8 +31,8 @@ import org.apache.tika.config.JsonConfig;
import org.apache.tika.exception.TikaException;
import org.apache.tika.io.EndianUtils;
import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.metadata.Property;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
@@ -61,8 +61,8 @@ public class DWGParser extends AbstractDWGParser {
}
public static String DWG_CUSTOM_META_PREFIX = "dwg-custom:";
- public static final PassthroughPrefix DWG_CUSTOM =
- PassthroughPrefix.file(DWG_CUSTOM_META_PREFIX, "DWG custom document properties");
+ public static final KeyPrefix DWG_CUSTOM =
+ KeyPrefix.file(DWG_CUSTOM_META_PREFIX, "DWG custom document properties");
/**
* Serial version UID
*/
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/ImageMetadataExtractor.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/ImageMetadataExtractor.java
index 4008c39910..872bda41eb 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/ImageMetadataExtractor.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/ImageMetadataExtractor.java
@@ -72,8 +72,8 @@ import org.xml.sax.SAXException;
import org.apache.tika.exception.TikaException;
import org.apache.tika.metadata.Geographic;
import org.apache.tika.metadata.IPTC;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.metadata.Property;
import org.apache.tika.metadata.TIFF;
import org.apache.tika.metadata.TikaCoreProperties;
@@ -93,10 +93,10 @@ public class ImageMetadataExtractor {
public static final String UNKNOWN_IMG_NS = "img" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
public static final String ICC_NS = "ICC" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
- public static final PassthroughPrefix UNKNOWN_IMG =
- PassthroughPrefix.file(UNKNOWN_IMG_NS, "unrecognized image tag names");
- public static final PassthroughPrefix ICC =
- PassthroughPrefix.file(ICC_NS, "unrecognized ICC profile tag names");
+ public static final KeyPrefix UNKNOWN_IMG =
+ KeyPrefix.file(UNKNOWN_IMG_NS, "unrecognized image tag names");
+ public static final KeyPrefix ICC =
+ KeyPrefix.file(ICC_NS, "unrecognized ICC profile tag names");
private final Metadata metadata;
private DirectoryHandler[] handlers;
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-mail-module/src/main/java/org/apache/tika/parser/mbox/MboxParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-mail-module/src/main/java/org/apache/tika/parser/mbox/MboxParser.java
index 16622cbc2b..4ce712b57a 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-mail-module/src/main/java/org/apache/tika/parser/mbox/MboxParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-mail-module/src/main/java/org/apache/tika/parser/mbox/MboxParser.java
@@ -41,9 +41,9 @@ import org.apache.tika.exception.TikaException;
import org.apache.tika.extractor.EmbeddedDocumentExtractor;
import org.apache.tika.extractor.EmbeddedDocumentUtil;
import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Message;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.metadata.Property;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
@@ -73,8 +73,8 @@ public class MboxParser implements Parser {
private static final Pattern EMAIL_ADDRESS_PATTERN = Pattern.compile("<(.*@.*)>");
private static final String EMAIL_HEADER_METADATA_PREFIX = "MboxParser-";
- private static final PassthroughPrefix EMAIL_HEADER =
- PassthroughPrefix.file(EMAIL_HEADER_METADATA_PREFIX, "mbox email header names");
+ private static final KeyPrefix EMAIL_HEADER =
+ KeyPrefix.file(EMAIL_HEADER_METADATA_PREFIX, "mbox email header names");
private static final String EMAIL_FROMLINE_METADATA = EMAIL_HEADER_METADATA_PREFIX + "from";
private final Map<Integer, Metadata> trackingMetadata = new HashMap<>();
private boolean tracking = false;
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/JackcessParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/JackcessParser.java
index 1d86b6fd98..71f60ef733 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/JackcessParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/JackcessParser.java
@@ -34,8 +34,8 @@ import org.apache.tika.exception.EncryptedDocumentException;
import org.apache.tika.exception.TikaException;
import org.apache.tika.exception.UnsupportedFormatException;
import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.metadata.Property;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
@@ -56,8 +56,8 @@ public class JackcessParser implements Parser {
public static final String SUMMARY_PROPERTY_PREFIX =
"mdb-summary-prop" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
- public static final PassthroughPrefix MDB_SUMMARY_PROPERTY =
- PassthroughPrefix.file(SUMMARY_PROPERTY_PREFIX, "MS Access summary properties");
+ public static final KeyPrefix MDB_SUMMARY_PROPERTY =
+ KeyPrefix.file(SUMMARY_PROPERTY_PREFIX, "MS Access summary properties");
private final static LinkResolver IGNORE_LINK_RESOLVER = new IgnoreLinkResolver();
private static final long serialVersionUID = -752276948656079347L;
private static final MediaType MEDIA_TYPE = MediaType.application("x-msaccess");
@@ -67,12 +67,12 @@ public class JackcessParser implements Parser {
// public static Property LINKED_DATABASES = Property.externalTextBag("LinkedDatabases");
public static String MDB_PROPERTY_PREFIX =
"mdb-prop" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
- public static final PassthroughPrefix MDB_PROPERTY =
- PassthroughPrefix.file(MDB_PROPERTY_PREFIX, "MS Access database properties");
+ public static final KeyPrefix MDB_PROPERTY =
+ KeyPrefix.file(MDB_PROPERTY_PREFIX, "MS Access database properties");
public static String USER_DEFINED_PROPERTY_PREFIX =
"mdb-user-prop" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
- public static final PassthroughPrefix MDB_USER_PROPERTY =
- PassthroughPrefix.file(USER_DEFINED_PROPERTY_PREFIX, "MS Access user-defined properties");
+ public static final KeyPrefix MDB_USER_PROPERTY =
+ KeyPrefix.file(USER_DEFINED_PROPERTY_PREFIX, "MS Access user-defined properties");
public static Property MDB_PW = Property.externalText("mdb:password");
private Locale locale = Locale.ROOT;
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-webarchive-module/src/main/java/org/apache/tika/parser/warc/WARCParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-webarchive-module/src/main/java/org/apache/tika/parser/warc/WARCParser.java
index 7206d6c7ca..9e5198d91d 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-webarchive-module/src/main/java/org/apache/tika/parser/warc/WARCParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-webarchive-module/src/main/java/org/apache/tika/parser/warc/WARCParser.java
@@ -39,8 +39,8 @@ import org.apache.tika.exception.WriteLimitReachedException;
import org.apache.tika.extractor.EmbeddedDocumentExtractor;
import org.apache.tika.extractor.EmbeddedDocumentUtil;
import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.metadata.Property;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.metadata.WARC;
@@ -65,10 +65,10 @@ public class WARCParser implements Parser {
public static String WARC_PREFIX = "warc:";
public static String WARC_HTTP_PREFIX = WARC_PREFIX + "http:";
- public static final PassthroughPrefix WARC_HEADER =
- PassthroughPrefix.file(WARC_PREFIX, "WARC record header names");
- public static final PassthroughPrefix WARC_HTTP_HEADER =
- PassthroughPrefix.file(WARC_HTTP_PREFIX, "WARC HTTP response header names");
+ public static final KeyPrefix WARC_HEADER =
+ KeyPrefix.file(WARC_PREFIX, "WARC record header names");
+ public static final KeyPrefix WARC_HTTP_HEADER =
+ KeyPrefix.file(WARC_HTTP_PREFIX, "WARC HTTP response header names");
public static String WARC_HTTP_STATUS = WARC_HTTP_PREFIX + "status";
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-xmp-commons/src/main/java/org/apache/tika/parser/xmp/XmpExtractor.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-xmp-commons/src/main/java/org/apache/tika/parser/xmp/XmpExtractor.java
index 3b96b3af0b..468f5c47cc 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-xmp-commons/src/main/java/org/apache/tika/parser/xmp/XmpExtractor.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-xmp-commons/src/main/java/org/apache/tika/parser/xmp/XmpExtractor.java
@@ -35,11 +35,11 @@ import org.xml.sax.SAXException;
import org.apache.tika.exception.TikaException;
import org.apache.tika.metadata.DublinCore;
import org.apache.tika.metadata.Google;
+import org.apache.tika.metadata.KeyPrefix;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.Office;
import org.apache.tika.metadata.PDF;
import org.apache.tika.metadata.PagedText;
-import org.apache.tika.metadata.PassthroughPrefix;
import org.apache.tika.metadata.Photoshop;
import org.apache.tika.metadata.Property;
import org.apache.tika.metadata.TIFF;
@@ -77,7 +77,7 @@ public class XmpExtractor {
// tk: field. Best-effort discovery surface: keys use the document's prefix (not the URI)
// and are non-contractual -- promote a field into TABLE when it needs a stable key.
static final String RAW_PREFIX = "xmp-raw:";
- static final PassthroughPrefix RAW = PassthroughPrefix.file(RAW_PREFIX, "unmapped XMP keys");
+ static final KeyPrefix RAW = KeyPrefix.file(RAW_PREFIX, "unmapped XMP keys");
// strip a trailing array index so a raw bag/seq is one multi-valued key, not foo:Bag[1], foo:Bag[2]
private static final Pattern TRAILING_INDEX = Pattern.compile("\\[\\d+\\]$");
(tika) 01/07: TIKA-4816 metadata-key stage 1: Property registration fix
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4816-metadata-key-api
in repository https://gitbox.apache.org/repos/asf/tika.git
commit eaa08478975411919901db1119849a0f913f50a2
Author: tallison <ta...@apache.org>
AuthorDate: Tue Aug 11 11:21:24 2026 -0400
TIKA-4816 metadata-key stage 1: Property registration fix
---
.../java/org/apache/tika/metadata/Property.java | 46 ++++++++-
.../org/apache/tika/metadata/PropertyTest.java | 113 +++++++++++++++++++++
2 files changed, 157 insertions(+), 2 deletions(-)
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/Property.java b/tika-core/src/main/java/org/apache/tika/metadata/Property.java
index 3d67141414..aa3df83307 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/Property.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/Property.java
@@ -25,6 +25,9 @@ import java.util.SortedSet;
import java.util.TreeSet;
import java.util.concurrent.ConcurrentHashMap;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
/**
* XMP property definition. Each instance of this class defines a single
* metadata property like "dc:format". In addition to the property name,
@@ -36,6 +39,7 @@ import java.util.concurrent.ConcurrentHashMap;
*/
public final class Property implements Comparable<Property> {
+ private static final Logger LOG = LoggerFactory.getLogger(Property.class);
private static final Map<String, Property> PROPERTIES = new ConcurrentHashMap<>();
private final String name;
private final boolean internal;
@@ -51,6 +55,20 @@ public final class Property implements Comparable<Property> {
private Property(String name, boolean internal, PropertyType propertyType, ValueType valueType,
String[] choices, Property primaryProperty,
Property[] secondaryExtractProperties) {
+ this(name, internal, propertyType, valueType, choices, primaryProperty,
+ secondaryExtractProperties, true);
+ }
+
+ /**
+ * @param register whether to intern this Property in the static registry. Composites
+ * (non-null primaryProperty) never register, regardless. Non-composite
+ * registration is first-wins: a name collision keeps the
+ * earlier-registered Property and this one is not stored; a
+ * same-name-different-shape collision also logs a WARN.
+ */
+ private Property(String name, boolean internal, PropertyType propertyType, ValueType valueType,
+ String[] choices, Property primaryProperty,
+ Property[] secondaryExtractProperties, boolean register) {
this.name = name;
this.internal = internal;
this.propertyType = propertyType;
@@ -70,8 +88,20 @@ public final class Property implements Comparable<Property> {
this.secondaryExtractProperties = null;
// Only store primary properties for lookup, not composites
- synchronized (PROPERTIES) {
- PROPERTIES.put(name, this);
+ if (register) {
+ synchronized (PROPERTIES) {
+ Property incumbent = PROPERTIES.putIfAbsent(name, this);
+ // same-shape re-mints (e.g. class re-init) stay quiet; a shape mismatch
+ // is a real bug (two differently-typed Properties claim the same name)
+ if (incumbent != null && (incumbent.propertyType != propertyType
+ || incumbent.valueType != valueType)) {
+ LOG.warn(
+ "Property registration collision for '{}': keeping {}/{}, " +
+ "dropping {}/{}",
+ name, incumbent.propertyType, incumbent.valueType, propertyType,
+ valueType);
+ }
+ }
}
}
}
@@ -94,6 +124,18 @@ public final class Property implements Comparable<Property> {
this(name, internal, propertyType, valueType, null);
}
+ /**
+ * Package-private, non-registering, lock-free minting path: skips {@code PROPERTIES}
+ * entirely, no interning and no lock. For call sites that construct Properties per-call
+ * from runtime/document-derived names, where interning would grow the static registry
+ * without bound (e.g. future {@code KeyPrefix} mints, a digest-template factory).
+ * Never composite.
+ */
+ static Property mintUnregistered(String name, boolean internal, PropertyType propertyType,
+ ValueType valueType, String[] choices) {
+ return new Property(name, internal, propertyType, valueType, choices, null, null, false);
+ }
+
/**
* Get the type of a property
*
diff --git a/tika-core/src/test/java/org/apache/tika/metadata/PropertyTest.java b/tika-core/src/test/java/org/apache/tika/metadata/PropertyTest.java
new file mode 100644
index 0000000000..74d24439af
--- /dev/null
+++ b/tika-core/src/test/java/org/apache/tika/metadata/PropertyTest.java
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.metadata;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertSame;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import org.junit.jupiter.api.Test;
+
+/** Registration semantics of {@link Property}: first-wins interning, and the non-registering mint path. */
+public class PropertyTest {
+
+ @Test
+ public void testFactoryRegistrationIsFirstWins() {
+ String name = "prop-test:collide-" + System.nanoTime();
+ Property first = Property.internalText(name);
+ Property second = Property.internalInteger(name);
+
+ // collision must not overwrite: the earlier-registered Property stays authoritative
+ assertSame(first, Property.get(name));
+ assertEquals(Property.ValueType.TEXT, Property.get(name).getValueType());
+ assertEquals(Property.ValueType.INTEGER, second.getValueType(),
+ "the losing Property object itself is still a valid, independent instance");
+ }
+
+ @Test
+ public void testShapeMismatchedCollisionStillFirstWins() {
+ // exercises the WARN branch (differing propertyType AND valueType); no log-capture
+ // idiom exists in this module, so this only asserts the first-wins outcome
+ String name = "prop-test:shape-mismatch-" + System.nanoTime();
+ Property first = Property.internalText(name);
+ Property second = Property.internalDateBag(name);
+
+ assertSame(first, Property.get(name));
+ assertEquals(Property.PropertyType.SIMPLE, Property.get(name).getPropertyType());
+ assertEquals(Property.PropertyType.BAG, second.getPropertyType());
+ }
+
+ @Test
+ public void testSameShapeReregistrationStaysQuietAndFirstWins() {
+ // exercises the non-WARN branch: identical shape, so no mismatch to report
+ String name = "prop-test:same-shape-" + System.nanoTime();
+ Property first = Property.internalText(name);
+ Property second = Property.internalText(name);
+
+ assertSame(first, Property.get(name));
+ assertFalse(first == second);
+ }
+
+ @Test
+ public void testUnregisteredPropertyIsFunctionalButNotInRegistry() {
+ String name = "prop-test:unregistered-" + System.nanoTime();
+ assertNull(Property.get(name), "precondition: name must not already be registered");
+
+ Property minted = Property.mintUnregistered(name, false, Property.PropertyType.SIMPLE,
+ Property.ValueType.TEXT, null);
+
+ assertEquals(name, minted.getName());
+ assertEquals(Property.PropertyType.SIMPLE, minted.getPropertyType());
+ assertEquals(Property.ValueType.TEXT, minted.getValueType());
+ assertFalse(minted.isInternal());
+ assertTrue(minted.isExternal());
+ assertSame(minted, minted.getPrimaryProperty());
+
+ // never interned: absent from both single lookup and prefix lookup
+ assertNull(Property.get(name));
+ assertFalse(Property.getProperties("prop-test").contains(minted));
+ }
+
+ @Test
+ public void testUnregisteredPropertyDoesNotBlockOrGetBlockedByFactoryRegistration() {
+ String name = "prop-test:coexist-" + System.nanoTime();
+
+ Property minted = Property.mintUnregistered(name, true, Property.PropertyType.SIMPLE,
+ Property.ValueType.TEXT, null);
+ assertNull(Property.get(name));
+
+ // a later factory call for the same name registers normally: the unregistered mint
+ // left no trace in PROPERTIES to collide with
+ Property registered = Property.internalText(name);
+ assertSame(registered, Property.get(name));
+ assertFalse(minted == registered);
+ }
+
+ @Test
+ public void testUnregisteredPropertyWithChoicesAndBagType() {
+ String name = "prop-test:bag-" + System.nanoTime();
+ Property minted = Property.mintUnregistered(name, true, Property.PropertyType.BAG,
+ Property.ValueType.CLOSED_CHOICE, new String[] {"a", "b"});
+
+ assertEquals(Property.PropertyType.BAG, minted.getPropertyType());
+ assertTrue(minted.isMultiValuePermitted());
+ assertTrue(minted.getChoices().contains("a"));
+ assertNull(Property.get(name));
+ }
+}
(tika) 02/07: TIKA-4816 metadata-key stage 2: mint-time reserved-name validation
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4816-metadata-key-api
in repository https://gitbox.apache.org/repos/asf/tika.git
commit ff467d33c5233669837e256b2de2eda45d24b7b0
Author: tallison <ta...@apache.org>
AuthorDate: Tue Aug 11 12:29:22 2026 -0400
TIKA-4816 metadata-key stage 2: mint-time reserved-name validation
---
.../apache/tika/digest/InputStreamDigester.java | 16 +-
.../java/org/apache/tika/metadata/Property.java | 171 +++++++++++++++++++++
.../java/org/apache/tika/metadata/Rendering.java | 18 ++-
.../apache/tika/metadata/TikaCoreProperties.java | 142 ++++++++++-------
.../org/apache/tika/metadata/TikaPagedText.java | 6 +-
.../java/org/apache/tika/utils/ParserUtils.java | 7 +-
.../tika/metadata/PropertyReservedNameTest.java | 161 +++++++++++++++++++
.../org/apache/tika/eval/app/ProfilerBase.java | 7 +-
.../apache/tika/metadata/metadata-key-fields.json | 3 +
.../tika/renderer/pdf/pdfbox/PDFBoxRenderer.java | 12 +-
10 files changed, 474 insertions(+), 69 deletions(-)
diff --git a/tika-core/src/main/java/org/apache/tika/digest/InputStreamDigester.java b/tika-core/src/main/java/org/apache/tika/digest/InputStreamDigester.java
index 849b54a833..842746c141 100644
--- a/tika-core/src/main/java/org/apache/tika/digest/InputStreamDigester.java
+++ b/tika-core/src/main/java/org/apache/tika/digest/InputStreamDigester.java
@@ -26,6 +26,7 @@ import java.util.concurrent.ConcurrentHashMap;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.Property;
+import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.utils.StringUtils;
@@ -35,6 +36,9 @@ import org.apache.tika.utils.StringUtils;
*/
public class InputStreamDigester implements Digester {
+ private static final String DIGEST_KEY_PREFIX =
+ TikaCoreProperties.TIKA_META_PREFIX + "digest" + TikaCoreProperties.NAMESPACE_PREFIX_DELIMITER;
+
private static final Map<String, Property> METADATA_KEY_PROPERTIES = new ConcurrentHashMap<>();
private final String algorithm;
@@ -51,10 +55,20 @@ public class InputStreamDigester implements Digester {
public InputStreamDigester(String algorithm, String metadataKey, Encoder encoder) {
this.algorithm = algorithm;
this.metadataProperty =
- METADATA_KEY_PROPERTIES.computeIfAbsent(metadataKey, Property::internalText);
+ METADATA_KEY_PROPERTIES.computeIfAbsent(metadataKey, InputStreamDigester::mintProperty);
this.encoder = encoder;
}
+ // metadataKey is config-supplied (algorithm/encoding names), not document-derived; the
+ // usual tk:digest:* shape mints via the curated template factory (stays registered, so
+ // Metadata.reconstruct/Property.get resolve it), anything else keeps the public factory.
+ private static Property mintProperty(String metadataKey) {
+ if (metadataKey.startsWith(DIGEST_KEY_PREFIX)) {
+ return TikaCoreProperties.digestProperty(metadataKey.substring(DIGEST_KEY_PREFIX.length()));
+ }
+ return Property.internalText(metadataKey);
+ }
+
private static void setContentLength(long length, Metadata metadata) {
if (StringUtils.isBlank(metadata.get(Metadata.CONTENT_LENGTH))) {
//only add it if it hasn't been populated already
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/Property.java b/tika-core/src/main/java/org/apache/tika/metadata/Property.java
index aa3df83307..da138f3a96 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/Property.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/Property.java
@@ -136,6 +136,34 @@ public final class Property implements Comparable<Property> {
return new Property(name, internal, propertyType, valueType, choices, null, null, false);
}
+ /**
+ * Guards the public factories: a {@code tk:}/{@code X-TIKA:} name can never be minted
+ * through them. Curated reserved constants go through the package-private
+ * {@code reservedInternal*}/{@code reservedExternal*} family below instead, so any live
+ * {@code Property} carrying a reserved name is, by construction, curated.
+ */
+ private static void requireNotReserved(String name) {
+ if (ReservedNamespaces.isTikaNative(name)) {
+ throw new IllegalArgumentException("'" + name + "' is in the reserved Tika-native "
+ + "namespace (tk:/X-TIKA:); it cannot be minted via a public Property "
+ + "factory. Curated tk: constants belong in org.apache.tika.metadata and "
+ + "mint via the package-private reserved* factories.");
+ }
+ }
+
+ /**
+ * Mirror image of {@link #requireNotReserved(String)} for the reserved factories below:
+ * fail loud (rather than silently minting a non-reserved Property through the reserved
+ * path) if fed a name that isn't actually reserved.
+ */
+ private static void requireReserved(String name) {
+ if (!ReservedNamespaces.isTikaNative(name)) {
+ throw new IllegalArgumentException("'" + name + "' is not in the reserved "
+ + "Tika-native namespace (tk:/X-TIKA:); use a public Property factory "
+ + "instead of the package-private reserved* path.");
+ }
+ }
+
/**
* Get the type of a property
*
@@ -175,90 +203,229 @@ public final class Property implements Comparable<Property> {
}
public static Property internalBoolean(String name) {
+ requireNotReserved(name);
return new Property(name, true, ValueType.BOOLEAN);
}
public static Property internalClosedChoise(String name, String... choices) {
+ requireNotReserved(name);
return new Property(name, true, ValueType.CLOSED_CHOICE, choices);
}
public static Property internalDate(String name) {
+ requireNotReserved(name);
return new Property(name, true, ValueType.DATE);
}
public static Property internalDateBag(String name) {
+ requireNotReserved(name);
return new Property(name, true, PropertyType.BAG, ValueType.DATE);
}
public static Property internalInteger(String name) {
+ requireNotReserved(name);
return new Property(name, true, ValueType.INTEGER);
}
public static Property internalIntegerSequence(String name) {
+ requireNotReserved(name);
return new Property(name, true, PropertyType.SEQ, ValueType.INTEGER);
}
public static Property internalRational(String name) {
+ requireNotReserved(name);
return new Property(name, true, ValueType.RATIONAL);
}
public static Property internalOpenChoise(String name, String... choices) {
+ requireNotReserved(name);
return new Property(name, true, ValueType.OPEN_CHOICE, choices);
}
public static Property internalReal(String name) {
+ requireNotReserved(name);
return new Property(name, true, ValueType.REAL);
}
public static Property internalText(String name) {
+ requireNotReserved(name);
return new Property(name, true, ValueType.TEXT);
}
public static Property internalTextBag(String name) {
+ requireNotReserved(name);
return new Property(name, true, PropertyType.BAG, ValueType.TEXT);
}
public static Property internalURI(String name) {
+ requireNotReserved(name);
return new Property(name, true, ValueType.URI);
}
public static Property externalClosedChoise(String name, String... choices) {
+ requireNotReserved(name);
return new Property(name, false, ValueType.CLOSED_CHOICE, choices);
}
public static Property externalOpenChoise(String name, String... choices) {
+ requireNotReserved(name);
return new Property(name, false, ValueType.OPEN_CHOICE, choices);
}
public static Property externalDate(String name) {
+ requireNotReserved(name);
return new Property(name, false, ValueType.DATE);
}
public static Property externalReal(String name) {
+ requireNotReserved(name);
return new Property(name, false, ValueType.REAL);
}
public static Property externalRealSeq(String name) {
+ requireNotReserved(name);
return new Property(name, false, PropertyType.SEQ, ValueType.REAL);
}
public static Property externalInteger(String name) {
+ requireNotReserved(name);
return new Property(name, false, ValueType.INTEGER);
}
public static Property externalBoolean(String name) {
+ requireNotReserved(name);
return new Property(name, false, ValueType.BOOLEAN);
}
public static Property externalBooleanSeq(String name) {
+ requireNotReserved(name);
return new Property(name, false, PropertyType.SEQ, ValueType.BOOLEAN);
}
public static Property externalText(String name) {
+ requireNotReserved(name);
return new Property(name, false, ValueType.TEXT);
}
public static Property externalTextBag(String name) {
+ requireNotReserved(name);
+ return new Property(name, false, PropertyType.BAG, ValueType.TEXT);
+ }
+
+ // ---- Package-private mirrors for curated tk:/X-TIKA: constants -------------------
+ // Same shapes as the public factories above, but (a) assert the name IS reserved
+ // (b) still register (curated constants must stay resolvable via Property.get /
+ // Metadata.reconstruct). Callers: TikaCoreProperties, TikaPagedText, Rendering — all
+ // in-package. Not for document-derived names; those mint unregistered (see
+ // mintUnregistered) via KeyPrefix (stage 3+).
+
+ static Property reservedInternalBoolean(String name) {
+ requireReserved(name);
+ return new Property(name, true, ValueType.BOOLEAN);
+ }
+
+ static Property reservedInternalClosedChoise(String name, String... choices) {
+ requireReserved(name);
+ return new Property(name, true, ValueType.CLOSED_CHOICE, choices);
+ }
+
+ static Property reservedInternalDate(String name) {
+ requireReserved(name);
+ return new Property(name, true, ValueType.DATE);
+ }
+
+ static Property reservedInternalDateBag(String name) {
+ requireReserved(name);
+ return new Property(name, true, PropertyType.BAG, ValueType.DATE);
+ }
+
+ static Property reservedInternalInteger(String name) {
+ requireReserved(name);
+ return new Property(name, true, ValueType.INTEGER);
+ }
+
+ static Property reservedInternalIntegerSequence(String name) {
+ requireReserved(name);
+ return new Property(name, true, PropertyType.SEQ, ValueType.INTEGER);
+ }
+
+ static Property reservedInternalRational(String name) {
+ requireReserved(name);
+ return new Property(name, true, ValueType.RATIONAL);
+ }
+
+ static Property reservedInternalOpenChoise(String name, String... choices) {
+ requireReserved(name);
+ return new Property(name, true, ValueType.OPEN_CHOICE, choices);
+ }
+
+ static Property reservedInternalReal(String name) {
+ requireReserved(name);
+ return new Property(name, true, ValueType.REAL);
+ }
+
+ static Property reservedInternalText(String name) {
+ requireReserved(name);
+ return new Property(name, true, ValueType.TEXT);
+ }
+
+ static Property reservedInternalTextBag(String name) {
+ requireReserved(name);
+ return new Property(name, true, PropertyType.BAG, ValueType.TEXT);
+ }
+
+ static Property reservedInternalURI(String name) {
+ requireReserved(name);
+ return new Property(name, true, ValueType.URI);
+ }
+
+ static Property reservedExternalClosedChoise(String name, String... choices) {
+ requireReserved(name);
+ return new Property(name, false, ValueType.CLOSED_CHOICE, choices);
+ }
+
+ static Property reservedExternalOpenChoise(String name, String... choices) {
+ requireReserved(name);
+ return new Property(name, false, ValueType.OPEN_CHOICE, choices);
+ }
+
+ static Property reservedExternalDate(String name) {
+ requireReserved(name);
+ return new Property(name, false, ValueType.DATE);
+ }
+
+ static Property reservedExternalReal(String name) {
+ requireReserved(name);
+ return new Property(name, false, ValueType.REAL);
+ }
+
+ static Property reservedExternalRealSeq(String name) {
+ requireReserved(name);
+ return new Property(name, false, PropertyType.SEQ, ValueType.REAL);
+ }
+
+ static Property reservedExternalInteger(String name) {
+ requireReserved(name);
+ return new Property(name, false, ValueType.INTEGER);
+ }
+
+ static Property reservedExternalBoolean(String name) {
+ requireReserved(name);
+ return new Property(name, false, ValueType.BOOLEAN);
+ }
+
+ static Property reservedExternalBooleanSeq(String name) {
+ requireReserved(name);
+ return new Property(name, false, PropertyType.SEQ, ValueType.BOOLEAN);
+ }
+
+ static Property reservedExternalText(String name) {
+ requireReserved(name);
+ return new Property(name, false, ValueType.TEXT);
+ }
+
+ static Property reservedExternalTextBag(String name) {
+ requireReserved(name);
return new Property(name, false, PropertyType.BAG, ValueType.TEXT);
}
@@ -267,6 +434,10 @@ public final class Property implements Comparable<Property> {
* <p>
* Note that name of the composite property is taken from its primary property,
* and primary and secondary properties must not be composite properties themselves.
+ * <p>
+ * No reserved-name check here: {@code primaryProperty} was already validated (or
+ * asserted reserved) at its own mint, and composites never register (see the
+ * constructor), so there is nothing new to forge or intern.
*
* @param primaryProperty
* @param secondaryExtractProperties
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/Rendering.java b/tika-core/src/main/java/org/apache/tika/metadata/Rendering.java
index 037dbb6a45..4eab31a2ac 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/Rendering.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/Rendering.java
@@ -19,6 +19,20 @@ package org.apache.tika.metadata;
public interface Rendering {
String RENDERING_PREFIX = "tk:rendering:";
- Property RENDERED_BY = Property.externalTextBag(RENDERING_PREFIX + "rendered-by");
- Property RENDERED_MS = Property.externalReal(RENDERING_PREFIX + "rendering-time-ms");
+ Property RENDERED_BY = Property.reservedExternalTextBag(RENDERING_PREFIX + "rendered-by");
+ Property RENDERED_MS = Property.reservedExternalReal(RENDERING_PREFIX + "rendering-time-ms");
+
+ /**
+ * Time PDFBox took to render the page to a {@code BufferedImage}.
+ * @see org.apache.tika.renderer.pdf.pdfbox.PDFBoxRenderer
+ */
+ Property PDFBOX_RENDERING_TIME_MS =
+ Property.reservedExternalReal(RENDERING_PREFIX + "pdfbox-rendering-ms");
+
+ /**
+ * Time PDFBox/java took to write the rendered image out (encoding cost varies by format).
+ * @see org.apache.tika.renderer.pdf.pdfbox.PDFBoxRenderer
+ */
+ Property PDFBOX_IMAGE_WRITING_TIME_MS =
+ Property.reservedExternalReal(RENDERING_PREFIX + "pdfbox-image-writing-ms");
}
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/TikaCoreProperties.java b/tika-core/src/main/java/org/apache/tika/metadata/TikaCoreProperties.java
index a977d8c377..0b63adc982 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/TikaCoreProperties.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/TikaCoreProperties.java
@@ -16,6 +16,8 @@
*/
package org.apache.tika.metadata;
+import org.apache.tika.utils.StringUtils;
+
/**
* Contains a core set of basic Tika metadata properties, which all parsers
* will attempt to supply (where the file format permits). These are all
@@ -58,7 +60,31 @@ public interface TikaCoreProperties {
* migrate to {@link #TIKA_META_PREFIX} ({@code tk:}) — see the 3.x-to-4.x metadata key table.
*/
String LEGACY_TIKA_META_PREFIX = "X-TIKA" + NAMESPACE_PREFIX_DELIMITER;
- Property EMBEDDED_DEPTH = Property.internalInteger(TIKA_META_PREFIX + "embedded-depth");
+
+ /**
+ * Template factory for digest keys: {@code tk:digest:<suffix>}, e.g.
+ * {@code tk:digest:SHA256} or {@code tk:digest:SHA256:BASE32} (algorithm[:encoding],
+ * config-supplied by a {@code Digester}, never document-derived). Validates only that
+ * {@code suffix} is non-empty and contains no whitespace — the shape constraint is
+ * hygiene, not security, since the input is trusted config.
+ *
+ * @since Apache Tika 4.0.0
+ */
+ static Property digestProperty(String suffix) {
+ if (StringUtils.isBlank(suffix)) {
+ throw new IllegalArgumentException("digest suffix must not be blank");
+ }
+ for (int i = 0; i < suffix.length(); i++) {
+ if (Character.isWhitespace(suffix.charAt(i))) {
+ throw new IllegalArgumentException(
+ "digest suffix must not contain whitespace: '" + suffix + "'");
+ }
+ }
+ return Property.reservedInternalText(
+ TIKA_META_PREFIX + "digest" + NAMESPACE_PREFIX_DELIMITER + suffix);
+ }
+
+ Property EMBEDDED_DEPTH = Property.reservedInternalInteger(TIKA_META_PREFIX + "embedded-depth");
/**
* This tracks the embedded file paths based on the name of embedded files
@@ -71,7 +97,7 @@ public interface TikaCoreProperties {
* For a more robust path, see {@link TikaCoreProperties#EMBEDDED_ID_PATH}.
*/
Property EMBEDDED_RESOURCE_PATH =
- Property.internalText(TIKA_META_PREFIX + "embedded-resource-path");
+ Property.reservedInternalText(TIKA_META_PREFIX + "embedded-resource-path");
/**
@@ -94,28 +120,28 @@ public interface TikaCoreProperties {
* For a more robust path, see {@link TikaCoreProperties#EMBEDDED_ID_PATH}.
*/
Property FINAL_EMBEDDED_RESOURCE_PATH =
- Property.internalText(TIKA_META_PREFIX + "final-embedded-resource-path");
+ Property.reservedInternalText(TIKA_META_PREFIX + "final-embedded-resource-path");
/**
* This tracks the embedded file paths based on the embedded file's
* {@link TikaCoreProperties#EMBEDDED_ID}.
*/
Property EMBEDDED_ID_PATH =
- Property.internalText(TIKA_META_PREFIX + "embedded-id-path");
+ Property.reservedInternalText(TIKA_META_PREFIX + "embedded-id-path");
/**
* This is a 1-index counter for embedded files, used by the RecursiveParserWrapper
*/
Property EMBEDDED_ID =
- Property.internalInteger(TIKA_META_PREFIX + "embedded-id");
+ Property.reservedInternalInteger(TIKA_META_PREFIX + "embedded-id");
- Property PARSE_TIME_MILLIS = Property.internalText(TIKA_META_PREFIX + "parse-time-millis");
+ Property PARSE_TIME_MILLIS = Property.reservedInternalText(TIKA_META_PREFIX + "parse-time-millis");
/**
* Simple class name of the content handler.
* @deprecated Use {@link #TIKA_CONTENT_HANDLER_TYPE} for the handler type enum value.
*/
@Deprecated
- Property TIKA_CONTENT_HANDLER = Property.internalText(TIKA_META_PREFIX + "content-handler");
+ Property TIKA_CONTENT_HANDLER = Property.reservedInternalText(TIKA_META_PREFIX + "content-handler");
/**
* The handler type used to produce {@link #TIKA_CONTENT}.
@@ -123,15 +149,15 @@ public interface TikaCoreProperties {
* enum name (e.g. {@code TEXT}, {@code MARKDOWN}, {@code HTML}, {@code XML}).
*/
Property TIKA_CONTENT_HANDLER_TYPE =
- Property.internalText(TIKA_META_PREFIX + "content-handler-type");
- Property TIKA_CONTENT = Property.internalText(TIKA_META_PREFIX + "content");
+ Property.reservedInternalText(TIKA_META_PREFIX + "content-handler-type");
+ Property TIKA_CONTENT = Property.reservedInternalText(TIKA_META_PREFIX + "content");
/**
* JSON array of chunks (text segments with optional embedding vectors and locators).
* Used by inference parsers and metadata filters to attach chunked representations
* of document content for downstream indexing and semantic search.
*/
- Property TIKA_CHUNKS = Property.internalText("tk:chunks");
+ Property TIKA_CHUNKS = Property.reservedInternalText("tk:chunks");
/**
* Use this to store parse exception information in the Metadata object.
*/
@@ -144,28 +170,36 @@ public interface TikaCoreProperties {
//exception in main file
Property CONTAINER_EXCEPTION =
- Property.internalText(TIKA_META_EXCEPTION_PREFIX + "container-exception");
+ Property.reservedInternalText(TIKA_META_EXCEPTION_PREFIX + "container-exception");
//exception in an embedded file
Property EMBEDDED_EXCEPTION =
- Property.internalTextBag(TIKA_META_EXCEPTION_PREFIX + "embedded-exception");
+ Property.reservedInternalTextBag(TIKA_META_EXCEPTION_PREFIX + "embedded-exception");
//exception handling the raw bytes of an embedded file by an EmbeddedDocumentByteStore
Property EMBEDDED_BYTES_EXCEPTION =
- Property.internalTextBag(TIKA_META_EXCEPTION_PREFIX + "embedded-bytes-exception");
+ Property.reservedInternalTextBag(TIKA_META_EXCEPTION_PREFIX + "embedded-bytes-exception");
//warning while parsing in an embedded file
Property EMBEDDED_WARNING =
- Property.internalTextBag(TIKA_META_EXCEPTION_PREFIX + "embedded-warning");
+ Property.reservedInternalTextBag(TIKA_META_EXCEPTION_PREFIX + "embedded-warning");
+
+ /**
+ * Identifies the embedded {@link org.apache.tika.parser.Parser} that produced
+ * {@link #EMBEDDED_EXCEPTION}, when known.
+ * @see org.apache.tika.utils.ParserUtils#EMBEDDED_PARSER the alias parser code uses
+ */
+ Property EMBEDDED_PARSER =
+ Property.reservedInternalText(TIKA_META_EXCEPTION_PREFIX + "embedded-parser");
Property WRITE_LIMIT_REACHED =
- Property.internalBoolean(TIKA_META_EXCEPTION_PREFIX + "write-limit-reached");
+ Property.reservedInternalBoolean(TIKA_META_EXCEPTION_PREFIX + "write-limit-reached");
Property EMBEDDED_RESOURCE_LIMIT_REACHED =
- Property.internalBoolean(TIKA_META_EXCEPTION_PREFIX + "embedded-resource-limit-reached");
+ Property.reservedInternalBoolean(TIKA_META_EXCEPTION_PREFIX + "embedded-resource-limit-reached");
Property EMBEDDED_DEPTH_LIMIT_REACHED =
- Property.internalBoolean(TIKA_META_EXCEPTION_PREFIX + "embedded-depth-limit-reached");
+ Property.reservedInternalBoolean(TIKA_META_EXCEPTION_PREFIX + "embedded-depth-limit-reached");
/**
* Use this to store exceptions caught during a parse that are
@@ -174,7 +208,7 @@ public interface TikaCoreProperties {
* a dependency.
*/
Property TIKA_META_EXCEPTION_WARNING =
- Property.internalTextBag(TIKA_META_EXCEPTION_PREFIX + "warn");
+ Property.reservedInternalTextBag(TIKA_META_EXCEPTION_PREFIX + "warn");
/**
* This means that metadata keys or metadata values were truncated.
@@ -182,7 +216,7 @@ public interface TikaCoreProperties {
* a field is not in the "include" set.
*/
Property TRUNCATED_METADATA =
- Property.internalBoolean(TIKA_META_WARN_PREFIX + "truncated-metadata");
+ Property.reservedInternalBoolean(TIKA_META_WARN_PREFIX + "truncated-metadata");
/**
* This indicates that only a portion of the file content was provided for detection.
@@ -190,7 +224,7 @@ public interface TikaCoreProperties {
* (e.g., not returning a detection result that requires reading to end of file).
*/
Property TRUNCATED_CONTENT_FOR_DETECTION =
- Property.internalBoolean(TIKA_META_PREFIX + "truncated-content-for-detection");
+ Property.reservedInternalBoolean(TIKA_META_PREFIX + "truncated-content-for-detection");
/**
* When content is truncated for detection, this stores the number of bytes
@@ -198,7 +232,7 @@ public interface TikaCoreProperties {
* to set appropriate mark limits.
*/
Property DETECTION_CONTENT_LENGTH =
- Property.internalInteger(TIKA_META_PREFIX + "detection-content-length");
+ Property.reservedInternalInteger(TIKA_META_PREFIX + "detection-content-length");
/**
* Use this to store exceptions caught while trying to read the
@@ -206,34 +240,34 @@ public interface TikaCoreProperties {
* a parse exception on the embedded resource.
*/
Property TIKA_META_EXCEPTION_EMBEDDED_STREAM =
- Property.internalTextBag(TIKA_META_EXCEPTION_PREFIX + "embedded-stream-exception");
- Property TIKA_PARSED_BY = Property.internalTextBag(TIKA_META_PREFIX + "parsed-by");
+ Property.reservedInternalTextBag(TIKA_META_EXCEPTION_PREFIX + "embedded-stream-exception");
+ Property TIKA_PARSED_BY = Property.reservedInternalTextBag(TIKA_META_PREFIX + "parsed-by");
/**
* Use this to store a record of all parsers that touched a given file
* in the container file's metadata.
*/
- Property TIKA_PARSED_BY_FULL_SET = Property.internalTextBag(TIKA_META_PREFIX + "parsed-by-full-set");
+ Property TIKA_PARSED_BY_FULL_SET = Property.reservedInternalTextBag(TIKA_META_PREFIX + "parsed-by-full-set");
- Property TIKA_DETECTED_LANGUAGE = Property.externalTextBag(TIKA_META_PREFIX +
+ Property TIKA_DETECTED_LANGUAGE = Property.reservedExternalTextBag(TIKA_META_PREFIX +
"detected-language");
- Property TIKA_DETECTED_LANGUAGE_CONFIDENCE = Property.externalTextBag(TIKA_META_PREFIX +
+ Property TIKA_DETECTED_LANGUAGE_CONFIDENCE = Property.reservedExternalTextBag(TIKA_META_PREFIX +
"detected-language-confidence");
- Property TIKA_DETECTED_LANGUAGE_CONFIDENCE_RAW = Property.externalRealSeq(TIKA_META_PREFIX +
+ Property TIKA_DETECTED_LANGUAGE_CONFIDENCE_RAW = Property.reservedExternalRealSeq(TIKA_META_PREFIX +
"detected-language-confidence-raw");
- Property RESOURCE_NAME_KEY = Property.internalText(TIKA_META_PREFIX + "resource-name");
+ Property RESOURCE_NAME_KEY = Property.reservedInternalText(TIKA_META_PREFIX + "resource-name");
/**
* Indicates that the file extension on the resource name was inferred by Tika
* (e.g., from content type detection) rather than provided by the original document.
*/
Property RESOURCE_NAME_EXTENSION_INFERRED =
- Property.externalBoolean(TIKA_META_PREFIX + "resource-name-extension-inferred");
+ Property.reservedExternalBoolean(TIKA_META_PREFIX + "resource-name-extension-inferred");
- Property EMBEDDED_RELATIONSHIP_ID = Property.internalText(TIKA_META_PREFIX + "embedded-relationship-id");
+ Property EMBEDDED_RELATIONSHIP_ID = Property.reservedInternalText(TIKA_META_PREFIX + "embedded-relationship-id");
String EMBEDDED_RESOURCE_TYPE_KEY = "tk:embedded-resource-type";
/**
@@ -242,7 +276,7 @@ public interface TikaCoreProperties {
* within the file.
*/
Property ORIGINAL_RESOURCE_NAME =
- Property.internalTextBag(TIKA_META_PREFIX + "orig-resource-name");
+ Property.reservedInternalTextBag(TIKA_META_PREFIX + "orig-resource-name");
/**
* This should be used to store the path (relative or full)
* of the source/container file, including the file name,
@@ -250,13 +284,13 @@ public interface TikaCoreProperties {
* <p>
* This can also be used for a primary key within a database.
*/
- Property SOURCE_PATH = Property.internalText(TIKA_META_PREFIX + "source-path");
+ Property SOURCE_PATH = Property.reservedInternalText(TIKA_META_PREFIX + "source-path");
/**
* This records the metadata as stored within a file for an embedded file's path
* including the file name. For example a zip file may include an msg with this path: /my-emails/important/this.msg
*/
- Property INTERNAL_PATH = Property.internalText(TIKA_META_PREFIX + "internal-path");
+ Property INTERNAL_PATH = Property.reservedInternalText(TIKA_META_PREFIX + "internal-path");
/**
* This is currently used to identify Content-Type that may be
@@ -265,25 +299,25 @@ public interface TikaCoreProperties {
* , or the value might come from outside the document. This information
* may be faulty and should be treated only as a hint.
*/
- Property CONTENT_TYPE_HINT = Property.internalText(TIKA_META_PREFIX + "content-type-hint");
+ Property CONTENT_TYPE_HINT = Property.reservedInternalText(TIKA_META_PREFIX + "content-type-hint");
/**
* This is used by users to override detection with the override detector.
*/
Property CONTENT_TYPE_USER_OVERRIDE =
- Property.internalText(TIKA_META_PREFIX + "content-type-override");
+ Property.reservedInternalText(TIKA_META_PREFIX + "content-type-override");
/**
* This is used by parsers to override detection of embedded resources
* with the override detector.
*/
Property CONTENT_TYPE_PARSER_OVERRIDE =
- Property.internalText(TIKA_META_PREFIX + "content-type-parser-override");
+ Property.reservedInternalText(TIKA_META_PREFIX + "content-type-parser-override");
/**
* This is set by DefaultDetector to store the result of MimeTypes (magic byte)
* detection. This allows downstream detectors to use it as a hint without
* re-running magic detection.
*/
Property CONTENT_TYPE_MAGIC_DETECTED =
- Property.internalText(TIKA_META_PREFIX + "content-type-magic-detected");
+ Property.reservedInternalText(TIKA_META_PREFIX + "content-type-magic-detected");
/**
* @see DublinCore#FORMAT
*/
@@ -396,7 +430,7 @@ public interface TikaCoreProperties {
* Java's {@link javax.imageio.ImageReader#getNumImages(boolean)}. See
* the javadocs for known limitations.
*/
- Property NUM_IMAGES = Property.internalInteger("tk:num-images");
+ Property NUM_IMAGES = Property.reservedInternalInteger("tk:num-images");
// Comment and rating properties
/**
@@ -406,21 +440,21 @@ public interface TikaCoreProperties {
/**
* Embedded resource type property
*/
- Property EMBEDDED_RESOURCE_TYPE = Property.internalClosedChoise(EMBEDDED_RESOURCE_TYPE_KEY,
+ Property EMBEDDED_RESOURCE_TYPE = Property.reservedInternalClosedChoise(EMBEDDED_RESOURCE_TYPE_KEY,
EmbeddedResourceType.ATTACHMENT.toString(), EmbeddedResourceType.INLINE.toString(),
EmbeddedResourceType.METADATA.toString(), EmbeddedResourceType.MACRO.toString(),
EmbeddedResourceType.THUMBNAIL.toString(), EmbeddedResourceType.RENDERING.toString());
- Property HAS_SIGNATURE = Property.internalBoolean("tk:has-signature");
+ Property HAS_SIGNATURE = Property.reservedInternalBoolean("tk:has-signature");
- Property SIGNATURE_NAME = Property.internalTextBag("tk:signature:name");
- Property SIGNATURE_DATE = Property.internalDateBag("tk:signature:date");
- Property SIGNATURE_LOCATION = Property.internalTextBag("tk:signature:location");
- Property SIGNATURE_REASON = Property.internalTextBag("tk:signature:reason");
- Property SIGNATURE_FILTER = Property.internalTextBag("tk:signature:filter");
- Property SIGNATURE_CONTACT_INFO = Property.internalTextBag("tk:signature:contact-info");
+ Property SIGNATURE_NAME = Property.reservedInternalTextBag("tk:signature:name");
+ Property SIGNATURE_DATE = Property.reservedInternalDateBag("tk:signature:date");
+ Property SIGNATURE_LOCATION = Property.reservedInternalTextBag("tk:signature:location");
+ Property SIGNATURE_REASON = Property.reservedInternalTextBag("tk:signature:reason");
+ Property SIGNATURE_FILTER = Property.reservedInternalTextBag("tk:signature:filter");
+ Property SIGNATURE_CONTACT_INFO = Property.reservedInternalTextBag("tk:signature:contact-info");
//is the file encrypted
- Property IS_ENCRYPTED = Property.internalBoolean(TIKA_META_PREFIX + "encrypted");
+ Property IS_ENCRYPTED = Property.reservedInternalBoolean(TIKA_META_PREFIX + "encrypted");
/**
* When an EncodingDetector detects an encoding, the encoding should be stored in this field.
@@ -429,14 +463,14 @@ public interface TikaCoreProperties {
* may choose to use a default encoding. We want to differentiate between a parser using a
* default encoding and the output of an EncodingDetector.
*/
- Property DETECTED_ENCODING = Property.externalText(TIKA_META_PREFIX + "detected-encoding");
+ Property DETECTED_ENCODING = Property.reservedExternalText(TIKA_META_PREFIX + "detected-encoding");
/**
* This should be the simple class name for the EncodingDetectors whose detected encoding
* was used in the parse.
*/
- Property ENCODING_DETECTOR = Property.externalText(TIKA_META_PREFIX + "encoding-detector");
+ Property ENCODING_DETECTOR = Property.reservedExternalText(TIKA_META_PREFIX + "encoding-detector");
/**
* Diagnostic trace showing which encoding detectors ran and what each returned,
@@ -444,7 +478,7 @@ public interface TikaCoreProperties {
* Example: {@code "HtmlEncodingDetector->UTF-8, Icu4jEncodingDetector->windows-1256 (scored)"}
*/
Property ENCODING_DETECTION_TRACE =
- Property.externalText(TIKA_META_PREFIX + "encoding-detection-trace");
+ Property.reservedExternalText(TIKA_META_PREFIX + "encoding-detection-trace");
/**
* The charset actually used to decode the stream when a superset override was applied.
@@ -456,13 +490,13 @@ public interface TikaCoreProperties {
* the same charset.
*/
Property DECODED_CHARSET =
- Property.externalText(TIKA_META_PREFIX + "decoded-charset");
+ Property.reservedExternalText(TIKA_META_PREFIX + "decoded-charset");
/**
* General metadata key for the count of non-final versions available within a file. This
* was added initially to support generalizing incremental updates in PDF.
*/
- Property VERSION_COUNT = Property.externalInteger(TIKA_META_PREFIX + "version-count");
+ Property VERSION_COUNT = Property.reservedExternalInteger(TIKA_META_PREFIX + "version-count");
/**
* General metadata key for the version number of a given file that contains
@@ -470,9 +504,9 @@ public interface TikaCoreProperties {
* The latest version does not have this metadata value. This was added initially
* to support generalizing incremental updates in PDF.
*/
- Property VERSION_NUMBER = Property.externalInteger(TIKA_META_PREFIX + "version-number");
+ Property VERSION_NUMBER = Property.reservedExternalInteger(TIKA_META_PREFIX + "version-number");
- Property PIPES_RESULT = Property.externalText(TIKA_META_PREFIX + "pipes-result");
+ Property PIPES_RESULT = Property.reservedExternalText(TIKA_META_PREFIX + "pipes-result");
/**
* A file might contain different types of embedded documents.
* The most common is the ATTACHMENT.
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/TikaPagedText.java b/tika-core/src/main/java/org/apache/tika/metadata/TikaPagedText.java
index 76faee52bd..0251e01fd0 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/TikaPagedText.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/TikaPagedText.java
@@ -30,7 +30,7 @@ public interface TikaPagedText {
* anchored to exactly one page; for resources spanning multiple pages
* see {@link #PAGE_NUMBERS}.
*/
- Property PAGE_NUMBER = Property.internalInteger(TIKA_PAGED_TEXT_PREFIX + "number");
+ Property PAGE_NUMBER = Property.reservedInternalInteger(TIKA_PAGED_TEXT_PREFIX + "number");
/**
* 1-based page numbers an embedded resource is anchored to, as a
@@ -56,7 +56,7 @@ public interface TikaPagedText {
* </ul>
*/
Property PAGE_NUMBERS =
- Property.internalIntegerSequence(TIKA_PAGED_TEXT_PREFIX + "numbers");
+ Property.reservedInternalIntegerSequence(TIKA_PAGED_TEXT_PREFIX + "numbers");
/**
* Sentinel value used as the sole element of {@link #PAGE_NUMBERS}
@@ -66,5 +66,5 @@ public interface TikaPagedText {
*/
int UNLINKED_PAGE = -1;
- Property PAGE_ROTATION = Property.internalRational(TIKA_PAGED_TEXT_PREFIX + "rotation");
+ Property PAGE_ROTATION = Property.reservedInternalRational(TIKA_PAGED_TEXT_PREFIX + "rotation");
}
diff --git a/tika-core/src/main/java/org/apache/tika/utils/ParserUtils.java b/tika-core/src/main/java/org/apache/tika/utils/ParserUtils.java
index 3536cf83e6..e194536076 100644
--- a/tika-core/src/main/java/org/apache/tika/utils/ParserUtils.java
+++ b/tika-core/src/main/java/org/apache/tika/utils/ParserUtils.java
@@ -31,8 +31,11 @@ import org.apache.tika.parser.ParserDecorator;
*/
public class ParserUtils {
- public final static Property EMBEDDED_PARSER = Property.internalText(
- TikaCoreProperties.TIKA_META_EXCEPTION_PREFIX + "embedded-parser");
+ /**
+ * @see TikaCoreProperties#EMBEDDED_PARSER the curated Property this aliases; {@code tk:}
+ * Properties mint only in org.apache.tika.metadata (see Property mint-time validation)
+ */
+ public final static Property EMBEDDED_PARSER = TikaCoreProperties.EMBEDDED_PARSER;
/**
diff --git a/tika-core/src/test/java/org/apache/tika/metadata/PropertyReservedNameTest.java b/tika-core/src/test/java/org/apache/tika/metadata/PropertyReservedNameTest.java
new file mode 100644
index 0000000000..dc1708f23b
--- /dev/null
+++ b/tika-core/src/test/java/org/apache/tika/metadata/PropertyReservedNameTest.java
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.metadata;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertSame;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+import java.util.LinkedHashMap;
+import java.util.Map;
+import java.util.function.Function;
+
+import org.junit.jupiter.api.Test;
+
+/**
+ * Stage 2 mint-time validation: the public {@code Property} factories reject reserved
+ * ({@code tk:}/{@code X-TIKA:}) names, curated reserved constants mint through the
+ * package-private path instead, and the digest template factory produces usable Properties.
+ */
+public class PropertyReservedNameTest {
+
+ // Every public static factory that mints a (non-composite) Property, keyed by method
+ // name for readable failure messages. Table-driven so one test method covers all 22
+ // shapes instead of 22 near-identical methods (no junit-jupiter-params in this module).
+ private static final Map<String, Function<String, Property>> PUBLIC_FACTORIES = publicFactories();
+
+ private static Map<String, Function<String, Property>> publicFactories() {
+ Map<String, Function<String, Property>> m = new LinkedHashMap<>();
+ m.put("internalBoolean", Property::internalBoolean);
+ m.put("internalClosedChoise", n -> Property.internalClosedChoise(n, "a", "b"));
+ m.put("internalDate", Property::internalDate);
+ m.put("internalDateBag", Property::internalDateBag);
+ m.put("internalInteger", Property::internalInteger);
+ m.put("internalIntegerSequence", Property::internalIntegerSequence);
+ m.put("internalRational", Property::internalRational);
+ m.put("internalOpenChoise", n -> Property.internalOpenChoise(n, "a", "b"));
+ m.put("internalReal", Property::internalReal);
+ m.put("internalText", Property::internalText);
+ m.put("internalTextBag", Property::internalTextBag);
+ m.put("internalURI", Property::internalURI);
+ m.put("externalClosedChoise", n -> Property.externalClosedChoise(n, "a", "b"));
+ m.put("externalOpenChoise", n -> Property.externalOpenChoise(n, "a", "b"));
+ m.put("externalDate", Property::externalDate);
+ m.put("externalReal", Property::externalReal);
+ m.put("externalRealSeq", Property::externalRealSeq);
+ m.put("externalInteger", Property::externalInteger);
+ m.put("externalBoolean", Property::externalBoolean);
+ m.put("externalBooleanSeq", Property::externalBooleanSeq);
+ m.put("externalText", Property::externalText);
+ m.put("externalTextBag", Property::externalTextBag);
+ return m;
+ }
+
+ @Test
+ public void testAllPublicFactoriesRejectTkPrefix() {
+ for (Map.Entry<String, Function<String, Property>> e : PUBLIC_FACTORIES.entrySet()) {
+ String label = e.getKey();
+ Function<String, Property> factory = e.getValue();
+ String name = "tk:prop-test-reserved-" + label + "-" + System.nanoTime();
+ assertThrows(IllegalArgumentException.class, () -> factory.apply(name),
+ label + " must reject a tk: name");
+ }
+ }
+
+ @Test
+ public void testAllPublicFactoriesRejectLegacyXTikaPrefix() {
+ for (Map.Entry<String, Function<String, Property>> e : PUBLIC_FACTORIES.entrySet()) {
+ String label = e.getKey();
+ Function<String, Property> factory = e.getValue();
+ String name = "X-TIKA:prop-test-reserved-" + label + "-" + System.nanoTime();
+ assertThrows(IllegalArgumentException.class, () -> factory.apply(name),
+ label + " must reject an X-TIKA: name");
+ }
+ }
+
+ @Test
+ public void testAllPublicFactoriesStillWorkForNonReservedNames() {
+ for (Map.Entry<String, Function<String, Property>> e : PUBLIC_FACTORIES.entrySet()) {
+ String label = e.getKey();
+ String name = "prop-test:reserved-ok-" + label + "-" + System.nanoTime();
+ Property p = e.getValue().apply(name);
+ assertEquals(name, p.getName(), label + " should still mint a non-reserved name");
+ assertSame(p, Property.get(name), label + " should still register");
+ }
+ }
+
+ @Test
+ public void testReservedFactoryRegistersAndResolvesViaPropertyGet() {
+ String name = "tk:prop-test-curated-" + System.nanoTime();
+ Property p = Property.reservedInternalText(name);
+ assertSame(p, Property.get(name), "curated tk: constants must stay in the global registry");
+ }
+
+ @Test
+ public void testReservedFactoryRejectsNonReservedName() {
+ // symmetry check: the reserved path must not silently accept a non-reserved name
+ String name = "prop-test:not-reserved-" + System.nanoTime();
+ assertThrows(IllegalArgumentException.class, () -> Property.reservedInternalText(name));
+ }
+
+ @Test
+ public void testDigestPropertyProducesUsableRegisteredProperty() {
+ String suffix = "SHA256-" + System.nanoTime();
+ Property p = TikaCoreProperties.digestProperty(suffix);
+
+ assertEquals("tk:digest:" + suffix, p.getName());
+ assertSame(p, Property.get(p.getName()), "digest Properties must register");
+
+ Metadata metadata = new Metadata();
+ metadata.set(p, "abc123");
+ assertEquals("abc123", metadata.get(p.getName()));
+ }
+
+ @Test
+ public void testDigestPropertyRejectsBlankOrNullSuffix() {
+ assertThrows(IllegalArgumentException.class, () -> TikaCoreProperties.digestProperty(""));
+ assertThrows(IllegalArgumentException.class, () -> TikaCoreProperties.digestProperty(null));
+ assertThrows(IllegalArgumentException.class, () -> TikaCoreProperties.digestProperty(" "));
+ }
+
+ @Test
+ public void testDigestPropertyRejectsWhitespaceInSuffix() {
+ assertThrows(IllegalArgumentException.class,
+ () -> TikaCoreProperties.digestProperty("SHA 256"));
+ }
+
+ @Test
+ public void testDigestPropertyAllowsColonSeparatedEncodingSuffix() {
+ // real shape produced by DigestDef.metadataKey(), e.g. "SHA256:BASE32"
+ String suffix = "SHA256:BASE32-" + System.nanoTime();
+ Property p = TikaCoreProperties.digestProperty(suffix);
+ assertEquals("tk:digest:" + suffix, p.getName());
+ }
+
+ @Test
+ public void testMintUnregisteredPathUnaffectedByReservedNameChecks() {
+ // mintUnregistered carries no reserved-name validation either way -- it is the
+ // non-registering path used by curated in-package code today and by KeyPrefix from
+ // stage 3; this documents that stage 2 left it untouched.
+ String name = "tk:prop-test-unregistered-" + System.nanoTime();
+ Property p = Property.mintUnregistered(name, true, Property.PropertyType.SIMPLE,
+ Property.ValueType.TEXT, null);
+ assertEquals(name, p.getName());
+ assertNull(Property.get(name), "mintUnregistered must never intern");
+ }
+}
diff --git a/tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ProfilerBase.java b/tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ProfilerBase.java
index 330455e610..8a26269383 100644
--- a/tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ProfilerBase.java
+++ b/tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ProfilerBase.java
@@ -71,7 +71,6 @@ import org.apache.tika.language.detect.LanguageResult;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.PDF;
import org.apache.tika.metadata.PagedText;
-import org.apache.tika.metadata.Property;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.ml.junkdetect.JunkDetector;
import org.apache.tika.pipes.api.fetcher.FetchKey;
@@ -86,8 +85,10 @@ public abstract class ProfilerBase {
protected static final AtomicInteger ID = new AtomicInteger();
static final long NON_EXISTENT_FILE_LENGTH = -1l;
final static int FILE_PATH_MAX_LEN = 1024;//max len for varchar for file_path
- //Container exception key from the 1.x branch
- private static final Property CONTAINER_EXCEPTION_1X = Property.externalText("X-TIKA" + ":EXCEPTION:runtime");
+ //Container exception key from the 1.x branch. Read-only lookup against legacy extract
+ //JSON, so a plain String key suffices -- Metadata.get(Property) is just get(name) anyway,
+ //and X-TIKA: is reserved: only org.apache.tika.metadata may mint a Property for it.
+ private static final String CONTAINER_EXCEPTION_1X = "X-TIKA" + ":EXCEPTION:runtime";
private static final Logger LOG = LoggerFactory.getLogger(ProfilerBase.class);
private static final String[] EXTRACT_EXTENSIONS = {".json", ".txt", ""};
private static final String[] COMPRESSION_EXTENSIONS = {"", ".bz2", ".gzip", ".zip",};
diff --git a/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-key-fields.json b/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-key-fields.json
index bba36567e8..94488c9af7 100644
--- a/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-key-fields.json
+++ b/tika-metadata-schema/src/main/resources/org/apache/tika/metadata/metadata-key-fields.json
@@ -387,6 +387,8 @@
{"class":"org.apache.tika.metadata.RTFMetadata","field":"EMBEDDED_ITEM","key":"rtf:embedded-item"},
{"class":"org.apache.tika.metadata.RTFMetadata","field":"EMBEDDED_TOPIC","key":"rtf:embedded-topic"},
{"class":"org.apache.tika.metadata.RTFMetadata","field":"THUMBNAIL","key":"rtf:thumbnail"},
+ {"class":"org.apache.tika.metadata.Rendering","field":"PDFBOX_IMAGE_WRITING_TIME_MS","key":"tk:rendering:pdfbox-image-writing-ms"},
+ {"class":"org.apache.tika.metadata.Rendering","field":"PDFBOX_RENDERING_TIME_MS","key":"tk:rendering:pdfbox-rendering-ms"},
{"class":"org.apache.tika.metadata.Rendering","field":"RENDERED_BY","key":"tk:rendering:rendered-by"},
{"class":"org.apache.tika.metadata.Rendering","field":"RENDERED_MS","key":"tk:rendering:rendering-time-ms"},
{"class":"org.apache.tika.metadata.TIFF","field":"BITS_PER_SAMPLE","key":"tiff:BitsPerSample"},
@@ -433,6 +435,7 @@
{"class":"org.apache.tika.metadata.TikaCoreProperties","field":"EMBEDDED_EXCEPTION","key":"tk:exception:embedded-exception"},
{"class":"org.apache.tika.metadata.TikaCoreProperties","field":"EMBEDDED_ID","key":"tk:embedded-id"},
{"class":"org.apache.tika.metadata.TikaCoreProperties","field":"EMBEDDED_ID_PATH","key":"tk:embedded-id-path"},
+ {"class":"org.apache.tika.metadata.TikaCoreProperties","field":"EMBEDDED_PARSER","key":"tk:exception:embedded-parser"},
{"class":"org.apache.tika.metadata.TikaCoreProperties","field":"EMBEDDED_RELATIONSHIP_ID","key":"tk:embedded-relationship-id"},
{"class":"org.apache.tika.metadata.TikaCoreProperties","field":"EMBEDDED_RESOURCE_LIMIT_REACHED","key":"tk:exception:embedded-resource-limit-reached"},
{"class":"org.apache.tika.metadata.TikaCoreProperties","field":"EMBEDDED_RESOURCE_PATH","key":"tk:embedded-resource-path"},
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRenderer.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRenderer.java
index fad55d468c..8c5c94592a 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRenderer.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/renderer/pdf/pdfbox/PDFBoxRenderer.java
@@ -64,17 +64,21 @@ public class PDFBoxRenderer implements PDDocumentRenderer {
/**
* This is the amount of time it takes for PDFBox to render the page
* to a BufferedImage
+ *
+ * @see Rendering#PDFBOX_RENDERING_TIME_MS the curated Property this aliases; {@code tk:}
+ * Properties mint only in org.apache.tika.metadata (see Property mint-time validation)
*/
- public static Property PDFBOX_RENDERING_TIME_MS =
- Property.externalReal(Rendering.RENDERING_PREFIX + "pdfbox-rendering-ms");
+ public static final Property PDFBOX_RENDERING_TIME_MS = Rendering.PDFBOX_RENDERING_TIME_MS;
/**
* This is the amount of time it takes for PDFBox/java to write the image after
* it has been rendered into a BufferedImage. Some formats take much longer
* to encode than others.
+ *
+ * @see Rendering#PDFBOX_IMAGE_WRITING_TIME_MS the curated Property this aliases
*/
- public static Property PDFBOX_IMAGE_WRITING_TIME_MS =
- Property.externalReal(Rendering.RENDERING_PREFIX + "pdfbox-image-writing-ms");
+ public static final Property PDFBOX_IMAGE_WRITING_TIME_MS =
+ Rendering.PDFBOX_IMAGE_WRITING_TIME_MS;
@Override
public Set<MediaType> getSupportedTypes(ParseContext context) {
(tika) 06/07: TIKA-4816 metadata-key stage 9: reconstruct negative-path tests + migration guide
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4816-metadata-key-api
in repository https://gitbox.apache.org/repos/asf/tika.git
commit bd803d799c406f6df96afdbc6953e3182bedfffa
Author: tallison <ta...@apache.org>
AuthorDate: Tue Aug 11 17:05:22 2026 -0400
TIKA-4816 metadata-key stage 9: reconstruct negative-path tests + migration guide
---
.../pages/migration-to-4x/metadata-changes-4x.adoc | 41 ++++++++++++++
.../pages/migration-to-4x/migrating-to-4x.adoc | 8 ++-
.../metadata/MetadataInternalKeyGuardTest.java | 37 ++++++++++++
.../UserMetadataReservedKeyStanceTest.java | 66 ++++++++++++++++++++++
4 files changed, 150 insertions(+), 2 deletions(-)
diff --git a/docs/modules/ROOT/pages/migration-to-4x/metadata-changes-4x.adoc b/docs/modules/ROOT/pages/migration-to-4x/metadata-changes-4x.adoc
index 1b5fada3aa..89a77fa3c9 100644
--- a/docs/modules/ROOT/pages/migration-to-4x/metadata-changes-4x.adoc
+++ b/docs/modules/ROOT/pages/migration-to-4x/metadata-changes-4x.adoc
@@ -347,6 +347,47 @@ String value = metadata.get("meta:some-property");
String value = metadata.get("office:some-property");
----
+== Write API Changes (reserved-key guard)
+
+The `tk:` namespace isn't just renamed in 4.0.0 -- writing to it changed shape. In 3.x, a
+`String` write to a reserved key (`X-TIKA:*`) simply succeeded: a document-controlled
+custom property literally named `X-TIKA:Parsed-By` could overwrite Tika's own computed
+value. In 4.0.0 that write throws instead, so the failure is loud rather than a silent
+security gap.
+
+=== For parser authors
+
+* Document-derived key names (scraped from the file: HTML `<meta>` names, custom
+ document properties, CSV headers, ...) must go through a `KeyPrefix`'s typed factory
+ methods (`text`, `textBag`, `date`, `integer`, `real`, `bool`) rather than a raw
+ `String` key. See xref:developers/metadata-keys.adoc[Adding a Metadata Key].
+* `Metadata#set(String, String)` / `#add(String, String)` throw `IllegalArgumentException`
+ in 4.0.0 when the name is a reserved `tk:` (or legacy `X-TIKA:`) key. In 3.x this write
+ succeeded; there was no guard.
+* `Property`'s public factories (`Property.externalText(...)`, `internalText(...)`, etc.)
+ throw `IllegalArgumentException` at construction if given a reserved name -- a curated
+ `tk:` `Property` can only be built from Tika's own package-private factories.
+* The throw is a `RuntimeException` (`IllegalArgumentException`), not a `TikaException`.
+ A harness that narrowly catches `TikaException` around a parse will see this as a new,
+ uncaught failure mode when an unmigrated parser hits a crafted file.
+
+=== For integrators
+
+* `metadata.set("tk:...", value)` -- and any other map-style write to a reserved key --
+ throws `IllegalArgumentException` in 4.0.0.
+* A manual copy loop (`for (String n : src.names()) dest.set(n, src.get(n))`) both
+ collapses multi-valued keys to their last value *and* throws on every `tk:` key that
+ every parse output already contains (`tk:parsed-by`, `tk:content`, ...). Replace it with
+ `Metadata#putAll(Metadata)`, which preserves multi-values and copies reserved keys
+ through their trusted route.
+* `Metadata#setAll(Properties)` is removed; there is no replacement -- it wrote a raw
+ map directly into `Metadata`, bypassing both the limiter and the reserved-key guard.
+ Use `putAll(Metadata)` or individual `set`/`add` calls instead.
+* `/pipes` and `/async` `userMetadata` behavior is unchanged: a request's own `tk:`-named
+ entries still land in that request's output. This is deliberate (the deserializer uses
+ `Metadata`'s trusted reconstruction route, not the guarded `String` route) and is
+ bounded to the requester's own request; it is not affected by the guard changes above.
+
== Rationale
The namespacing of metadata keys provides several benefits:
diff --git a/docs/modules/ROOT/pages/migration-to-4x/migrating-to-4x.adoc b/docs/modules/ROOT/pages/migration-to-4x/migrating-to-4x.adoc
index a4d178eb59..2cb8df0c65 100644
--- a/docs/modules/ROOT/pages/migration-to-4x/migrating-to-4x.adoc
+++ b/docs/modules/ROOT/pages/migration-to-4x/migrating-to-4x.adoc
@@ -180,10 +180,11 @@ complete documentation of all available options.
== Metadata Key Changes
Tika 4.x prefixes all "user generated" metadata keys to prevent overwrites and improve
-namespace clarity.
+namespace clarity. Writing to a reserved `tk:` key by `String` name also changed: it
+silently succeeded in 3.x and now throws.
See xref:migration-to-4x/metadata-changes-4x.adoc[Metadata Changes in 4.x] for complete details, including
-a full table of changes and code migration examples.
+a full table of changes, the write-API/reserved-key-guard changes, and code migration examples.
== API Changes
@@ -244,6 +245,9 @@ for details.
== Deprecations and Removals
* `TikaConfig` -- replaced by `TikaLoader`
+* `Metadata#setAll(Properties)` -- raw map write that bypassed the reserved-key guard;
+ use `Metadata#putAll(Metadata)` instead (see
+ xref:migration-to-4x/metadata-changes-4x.adoc[Metadata Changes in 4.x])
* `CompositeExternalParser` -- external parsers now require explicit JSON configuration
* `ExternalParsersFactory` and XML-based external parser auto-discovery
* DOM-based OOXML extractors (`XWPFWordExtractorDecorator`, `XSLFPowerPointExtractorDecorator`)
diff --git a/tika-core/src/test/java/org/apache/tika/metadata/MetadataInternalKeyGuardTest.java b/tika-core/src/test/java/org/apache/tika/metadata/MetadataInternalKeyGuardTest.java
index 8603c5c27f..6bda921d26 100644
--- a/tika-core/src/test/java/org/apache/tika/metadata/MetadataInternalKeyGuardTest.java
+++ b/tika-core/src/test/java/org/apache/tika/metadata/MetadataInternalKeyGuardTest.java
@@ -139,4 +139,41 @@ public class MetadataInternalKeyGuardTest {
assertArrayEquals(new String[] {"p1", "p2"},
metadata.getValues(TikaCoreProperties.TIKA_PARSED_BY));
}
+
+ @Test
+ public void testReconstructNonReservedRoutesThroughStringPath() {
+ Metadata metadata = new Metadata();
+ metadata.reconstruct("my:customKey", "v1", false);
+ assertEquals("v1", metadata.get("my:customKey"));
+
+ metadata.reconstruct("my:customKey", "v2", true);
+ assertArrayEquals(new String[] {"v1", "v2"}, metadata.getValues("my:customKey"));
+ }
+
+ /**
+ * Design doc "Honest framing": {@code reconstruct} is a deliberately trusted route,
+ * not subject to the String-route guard -- for both a reserved name with a registered
+ * curated Property and one with none. Contrasts directly against the drop asserted by
+ * {@link #testStringWriteToInternalKeyIsDropped()} /
+ * {@link #testReconstructPreservesUnregisteredReservedKey()} on the same names, so a
+ * regression that made {@code reconstruct} start dropping (or the guard start
+ * exempting it) would be caught here either way.
+ */
+ @Test
+ public void testReconstructIsNotSubjectToReservedKeyDrop() {
+ Metadata metadata = new Metadata();
+
+ // registered curated Property
+ metadata.set(TikaCoreProperties.TIKA_CONTENT.getName(), "dropped-by-guard");
+ assertNull(metadata.get(TikaCoreProperties.TIKA_CONTENT));
+ metadata.reconstruct(TikaCoreProperties.TIKA_CONTENT.getName(), "lands-via-reconstruct", false);
+ assertEquals("lands-via-reconstruct", metadata.get(TikaCoreProperties.TIKA_CONTENT));
+
+ // reserved but unregistered
+ String unregistered = TikaCoreProperties.TIKA_META_PREFIX + "noSuchRegisteredProperty2";
+ metadata.set(unregistered, "dropped-by-guard");
+ assertNull(metadata.get(unregistered));
+ metadata.reconstruct(unregistered, "lands-via-reconstruct", false);
+ assertEquals("lands-via-reconstruct", metadata.get(unregistered));
+ }
}
diff --git a/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/serialization/UserMetadataReservedKeyStanceTest.java b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/serialization/UserMetadataReservedKeyStanceTest.java
new file mode 100644
index 0000000000..86c97179b2
--- /dev/null
+++ b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/serialization/UserMetadataReservedKeyStanceTest.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.pipes.core.serialization;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+import java.io.IOException;
+import java.io.StringReader;
+
+import org.junit.jupiter.api.Test;
+
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.metadata.TikaCoreProperties;
+import org.apache.tika.pipes.api.FetchEmitTuple;
+
+/**
+ * Pins the accepted stance from the metadata-key-api design doc's "Honest framing"
+ * section: {@code FetchEmitTupleDeserializer} builds a tuple's userMetadata via
+ * {@link Metadata#reconstruct}, a deliberately trusted, request-reachable route -- so a
+ * requester CAN assert Tika's reserved {@code tk:} keys (e.g. {@code tk:content}) in
+ * their own request's userMetadata, and that value survives deserialization verbatim.
+ * This is accepted, not a bug: the requester only poisons their own request's output;
+ * deployment-level authn/authz on who may submit tuples is the actual trust boundary,
+ * not this deserializer (see design doc, "Honest framing"). If this test starts failing
+ * because the reserved key silently drops instead of landing, that is a stance change
+ * that must be re-approved, not "fixed."
+ */
+public class UserMetadataReservedKeyStanceTest {
+
+ @Test
+ public void userMetadataCanAssertReservedKeys_acceptedStance() throws IOException {
+ String json = """
+ {
+ "id": "id1",
+ "fetcher": "fs",
+ "fetchKey": "fetchKey1",
+ "metadata": {
+ "tk:content": "attacker-injected",
+ "tk:noSuchRegisteredProperty": "also-injected"
+ }
+ }
+ """;
+
+ FetchEmitTuple t = JsonFetchEmitTuple.fromJson(new StringReader(json));
+ Metadata metadata = t.getMetadata();
+
+ // registered curated Property: reconstruct routes through it, so it lands
+ assertEquals("attacker-injected", metadata.get(TikaCoreProperties.TIKA_CONTENT));
+ // unregistered reserved name: reconstruct's trusted-write branch, still lands
+ assertEquals("also-injected", metadata.get("tk:noSuchRegisteredProperty"));
+ }
+}
(tika) 04/07: TIKA-4816 metadata-key stage 4: putAll(Metadata) public copy API
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4816-metadata-key-api
in repository https://gitbox.apache.org/repos/asf/tika.git
commit 24606fcd3bded3008f244b7b674a1be377080420
Author: tallison <ta...@apache.org>
AuthorDate: Tue Aug 11 14:22:02 2026 -0400
TIKA-4816 metadata-key stage 4: putAll(Metadata) public copy API
---
.../java/org/apache/tika/metadata/Metadata.java | 34 ++++++++
.../java/org/apache/tika/utils/ParserUtils.java | 11 +--
.../apache/tika/metadata/MetadataPutAllTest.java | 98 ++++++++++++++++++++++
3 files changed, 133 insertions(+), 10 deletions(-)
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/Metadata.java b/tika-core/src/main/java/org/apache/tika/metadata/Metadata.java
index 1f20a5ce09..9ea8385850 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/Metadata.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/Metadata.java
@@ -361,6 +361,40 @@ public class Metadata
}
}
+ /**
+ * Copies every key from {@code other} into this Metadata: for each name in
+ * {@code other}, this Metadata's values for that name are replaced wholesale with
+ * {@code other}'s values, in order (multi-values preserved); names absent from
+ * {@code other} are left untouched. Each value is written via
+ * {@link #reconstruct(String, String, boolean)}, so reserved {@code tk:} keys copy
+ * through their trusted route rather than the String-guarded route.
+ * <p>
+ * This is the supported replacement for a manual
+ * {@code for (String n : src.names()) dest.set(n, src.get(n))} copy loop, which
+ * silently collapses multi-valued keys to a single value today and throws on
+ * {@code tk:}-prefixed keys once the reserved-key guard is flipped from drop to throw.
+ *
+ * @param other the Metadata to copy from; {@code other == this} is a no-op
+ * @throws NullPointerException if other is null
+ * @since Apache Tika 4.0
+ */
+ public void putAll(Metadata other) {
+ Objects.requireNonNull(other, "other must not be null");
+ if (other == this) {
+ return;
+ }
+ for (String n : other.names()) {
+ String[] vals = other.getValues(n);
+ if (vals.length == 0) {
+ continue;
+ }
+ reconstruct(n, vals[0], false);
+ for (int i = 1; i < vals.length; i++) {
+ reconstruct(n, vals[i], true);
+ }
+ }
+ }
+
/**
* Add a metadata name/value mapping. Add the specified value to the list of
* values associated to the specified metadata name.
diff --git a/tika-core/src/main/java/org/apache/tika/utils/ParserUtils.java b/tika-core/src/main/java/org/apache/tika/utils/ParserUtils.java
index e194536076..bf1a845247 100644
--- a/tika-core/src/main/java/org/apache/tika/utils/ParserUtils.java
+++ b/tika-core/src/main/java/org/apache/tika/utils/ParserUtils.java
@@ -43,16 +43,7 @@ public class ParserUtils {
*/
public static Metadata cloneMetadata(Metadata m) {
Metadata clone = new Metadata();
-
- for (String n : m.names()) {
- if (!m.isMultiValued(n)) {
- clone.reconstruct(n, m.get(n), false);
- } else {
- for (String val : m.getValues(n)) {
- clone.reconstruct(n, val, true);
- }
- }
- }
+ clone.putAll(m);
return clone;
}
diff --git a/tika-core/src/test/java/org/apache/tika/metadata/MetadataPutAllTest.java b/tika-core/src/test/java/org/apache/tika/metadata/MetadataPutAllTest.java
new file mode 100644
index 0000000000..003eda4585
--- /dev/null
+++ b/tika-core/src/test/java/org/apache/tika/metadata/MetadataPutAllTest.java
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.metadata;
+
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+import org.junit.jupiter.api.Test;
+
+/** {@link Metadata#putAll(Metadata)}: the provenance-preserving per-key-replace copy API. */
+public class MetadataPutAllTest {
+
+ @Test
+ public void testMultiValuedKeyCopiesAllValuesInOrder() {
+ Metadata src = new Metadata();
+ src.add("multi", "a");
+ src.add("multi", "b");
+ src.add("multi", "c");
+
+ Metadata dest = new Metadata();
+ dest.putAll(src);
+
+ // the manual `dest.set(n, src.get(n))` loop this replaces would collapse to just "a"
+ assertArrayEquals(new String[] {"a", "b", "c"}, dest.getValues("multi"));
+ }
+
+ @Test
+ public void testReservedKeyCopiesThroughTrustedRoute() {
+ Metadata src = new Metadata();
+ src.reconstruct(TikaCoreProperties.TIKA_CONTENT.getName(), "the content", false);
+
+ Metadata dest = new Metadata();
+ dest.putAll(src);
+
+ assertEquals("the content", dest.get(TikaCoreProperties.TIKA_CONTENT));
+ }
+
+ @Test
+ public void testPerKeyReplaceSemantics() {
+ Metadata src = new Metadata();
+ src.add("k", "only");
+
+ Metadata dest = new Metadata();
+ dest.add("k", "v1");
+ dest.add("k", "v2");
+ dest.add("k", "v3");
+
+ dest.putAll(src);
+
+ assertArrayEquals(new String[] {"only"}, dest.getValues("k"));
+ }
+
+ @Test
+ public void testKeysAbsentFromOtherAreUntouched() {
+ Metadata src = new Metadata();
+ src.add("inOther", "x");
+
+ Metadata dest = new Metadata();
+ dest.add("onlyInDest", "keepme");
+
+ dest.putAll(src);
+
+ assertEquals("keepme", dest.get("onlyInDest"));
+ assertEquals("x", dest.get("inOther"));
+ }
+
+ @Test
+ public void testSelfPutAllIsNoOp() {
+ Metadata metadata = new Metadata();
+ metadata.add("k", "v1");
+ metadata.add("k", "v2");
+
+ metadata.putAll(metadata);
+
+ assertArrayEquals(new String[] {"v1", "v2"}, metadata.getValues("k"));
+ }
+
+ @Test
+ public void testNullOtherThrowsNPE() {
+ Metadata metadata = new Metadata();
+ assertThrows(NullPointerException.class, () -> metadata.putAll(null));
+ }
+}
(tika) 07/07: TIKA-4816 fix EMBEDDED_EXCEPTION erasure under active metadata limiter
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4816-metadata-key-api
in repository https://gitbox.apache.org/repos/asf/tika.git
commit 0e3fe950599a828c9676d22dbcee8ace4b0cca19
Author: tallison <ta...@apache.org>
AuthorDate: Wed Aug 12 11:41:30 2026 -0400
TIKA-4816 fix EMBEDDED_EXCEPTION erasure under active metadata limiter
---
.../writefilter/StandardMetadataLimiter.java | 3 +-
.../writefilter/StandardMetadataLimiterTest.java | 75 ++++++++++++++++++++++
2 files changed, 77 insertions(+), 1 deletion(-)
diff --git a/tika-core/src/main/java/org/apache/tika/metadata/writefilter/StandardMetadataLimiter.java b/tika-core/src/main/java/org/apache/tika/metadata/writefilter/StandardMetadataLimiter.java
index 913057a8f0..4acd9db81a 100644
--- a/tika-core/src/main/java/org/apache/tika/metadata/writefilter/StandardMetadataLimiter.java
+++ b/tika-core/src/main/java/org/apache/tika/metadata/writefilter/StandardMetadataLimiter.java
@@ -82,12 +82,13 @@ public class StandardMetadataLimiter implements MetadataWriteLimiter, Serializab
ALWAYS_SET_FIELDS.add(AccessPermissions.EXTRACT_FOR_ACCESSIBILITY.getName());
ALWAYS_SET_FIELDS.add(Metadata.CONTENT_DISPOSITION.getName());
ALWAYS_SET_FIELDS.add(TikaCoreProperties.CONTAINER_EXCEPTION.getName());
- ALWAYS_SET_FIELDS.add(TikaCoreProperties.EMBEDDED_EXCEPTION.getName());
//Metadata.CONTENT_LOCATION? used by the html parser
}
static {
ALWAYS_ADD_FIELDS.add(TikaCoreProperties.TIKA_PARSED_BY.getName());
+ //bag-typed: one embedded-file failure per add() call must not clobber prior ones
+ ALWAYS_ADD_FIELDS.add(TikaCoreProperties.EMBEDDED_EXCEPTION.getName());
}
private static final String METADATA_TRUNCATED_KEY =
diff --git a/tika-serialization/src/test/java/org/apache/tika/metadata/writefilter/StandardMetadataLimiterTest.java b/tika-serialization/src/test/java/org/apache/tika/metadata/writefilter/StandardMetadataLimiterTest.java
index 989d1cdf11..9ad85c5fb2 100644
--- a/tika-serialization/src/test/java/org/apache/tika/metadata/writefilter/StandardMetadataLimiterTest.java
+++ b/tika-serialization/src/test/java/org/apache/tika/metadata/writefilter/StandardMetadataLimiterTest.java
@@ -286,6 +286,81 @@ public class StandardMetadataLimiterTest extends TikaTest {
}
+ @Test
+ public void testEmbeddedExceptionAddRetainsAllValues() throws Exception {
+ //EMBEDDED_EXCEPTION is bag-typed and in ALWAYS_ADD_FIELDS: N adds must retain
+ //N values, in order, even under an active limiter
+ Metadata metadata = filter(100, 10000, 10000, 100,
+ Collections.EMPTY_SET, Collections.EMPTY_SET, true);
+ metadata.add(TikaCoreProperties.EMBEDDED_EXCEPTION, "exception 1");
+ metadata.add(TikaCoreProperties.EMBEDDED_EXCEPTION, "exception 2");
+ metadata.add(TikaCoreProperties.EMBEDDED_EXCEPTION, "exception 3");
+
+ assertArrayEquals(new String[]{"exception 1", "exception 2", "exception 3"},
+ metadata.getValues(TikaCoreProperties.EMBEDDED_EXCEPTION));
+ }
+
+ @Test
+ public void testEmbeddedExceptionSetStillReplaces() throws Exception {
+ //set() must still replace, even though add() now appends
+ Metadata metadata = filter(100, 10000, 10000, 100,
+ Collections.EMPTY_SET, Collections.EMPTY_SET, true);
+ metadata.add(TikaCoreProperties.EMBEDDED_EXCEPTION, "exception 1");
+ metadata.add(TikaCoreProperties.EMBEDDED_EXCEPTION, "exception 2");
+ metadata.set(TikaCoreProperties.EMBEDDED_EXCEPTION, "replacement");
+
+ assertArrayEquals(new String[]{"replacement"},
+ metadata.getValues(TikaCoreProperties.EMBEDDED_EXCEPTION));
+ }
+
+ @Test
+ public void testEmbeddedExceptionPerValueTruncation() throws Exception {
+ //per-value truncation and the truncated-metadata marker must still apply,
+ //without erasing values already added
+ Metadata metadata = filter(100, 10, 100000, 100,
+ Collections.EMPTY_SET, Collections.EMPTY_SET, true);
+ metadata.add(TikaCoreProperties.EMBEDDED_EXCEPTION, "short");
+ //800 utf-16 bytes, over the 300-byte always-field floor
+ String longValue = "x".repeat(400);
+ metadata.add(TikaCoreProperties.EMBEDDED_EXCEPTION, longValue);
+
+ String[] values = metadata.getValues(TikaCoreProperties.EMBEDDED_EXCEPTION);
+ assertEquals(2, values.length);
+ assertEquals("short", values[0]);
+ //truncated to the 300-byte (utf-16) always-field floor = 150 chars
+ assertEquals(150, values[1].length());
+ assertTruncated(metadata);
+ }
+
+ @Test
+ public void testEmbeddedExceptionSurvivesOverBudget() throws Exception {
+ //always-add fields must not be dropped even once the overall budget is exhausted
+ Metadata metadata = filter(100, 1, 1, 100,
+ Collections.EMPTY_SET, Collections.EMPTY_SET, true);
+ metadata.add(TikaCoreProperties.EMBEDDED_EXCEPTION, "e1");
+ metadata.add(TikaCoreProperties.EMBEDDED_EXCEPTION, "e2");
+ metadata.add(TikaCoreProperties.EMBEDDED_EXCEPTION, "e3");
+
+ assertArrayEquals(new String[]{"e1", "e2", "e3"},
+ metadata.getValues(TikaCoreProperties.EMBEDDED_EXCEPTION));
+ }
+
+ @Test
+ public void testSimpleAlwaysSetFieldStillReplacesOnAdd() throws Exception {
+ //CONTAINER_EXCEPTION is SIMPLE (not bag) and stays in ALWAYS_SET_FIELDS:
+ //the limiter's add() dispatch must keep replacing, as before this fix.
+ //Use addTrusted() to reach the limiter directly: Metadata.add(Property,..) itself
+ //already throws PropertyTypeException on a 2nd add to a SIMPLE property, so that
+ //typed path can't otherwise exercise the limiter's repeated-add behavior.
+ Metadata metadata = filter(100, 10000, 10000, 100,
+ Collections.EMPTY_SET, Collections.EMPTY_SET, true);
+ metadata.addTrusted(TikaCoreProperties.CONTAINER_EXCEPTION.getName(), "container exception 1");
+ metadata.addTrusted(TikaCoreProperties.CONTAINER_EXCEPTION.getName(), "container exception 2");
+
+ assertArrayEquals(new String[]{"container exception 2"},
+ metadata.getValues(TikaCoreProperties.CONTAINER_EXCEPTION));
+ }
+
@Test
public void testExclude() throws Exception {
TikaLoader loader = TikaLoader.load(getConfigPath(getClass(), "TIKA-3695-exclude.json"));