You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tika.apache.org by ta...@apache.org on 2026/08/12 16:03:08 UTC
(tika) 06/07: TIKA-4809 metadata-key stage 9: reconstruct negative-path tests + migration guide
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4809-metadata-key-api
in repository https://gitbox.apache.org/repos/asf/tika.git
commit 149bae1c613629581cebb04ce7606ff25228a800
Author: tallison <ta...@apache.org>
AuthorDate: Tue Aug 11 17:05:22 2026 -0400
TIKA-4809 metadata-key stage 9: reconstruct negative-path tests + migration guide
---
.../pages/migration-to-4x/metadata-changes-4x.adoc | 41 ++++++++++++++
.../pages/migration-to-4x/migrating-to-4x.adoc | 8 ++-
.../metadata/MetadataInternalKeyGuardTest.java | 37 ++++++++++++
.../UserMetadataReservedKeyStanceTest.java | 66 ++++++++++++++++++++++
4 files changed, 150 insertions(+), 2 deletions(-)
diff --git a/docs/modules/ROOT/pages/migration-to-4x/metadata-changes-4x.adoc b/docs/modules/ROOT/pages/migration-to-4x/metadata-changes-4x.adoc
index 1b5fada3aa..89a77fa3c9 100644
--- a/docs/modules/ROOT/pages/migration-to-4x/metadata-changes-4x.adoc
+++ b/docs/modules/ROOT/pages/migration-to-4x/metadata-changes-4x.adoc
@@ -347,6 +347,47 @@ String value = metadata.get("meta:some-property");
String value = metadata.get("office:some-property");
----
+== Write API Changes (reserved-key guard)
+
+The `tk:` namespace isn't just renamed in 4.0.0 -- writing to it changed shape. In 3.x, a
+`String` write to a reserved key (`X-TIKA:*`) simply succeeded: a document-controlled
+custom property literally named `X-TIKA:Parsed-By` could overwrite Tika's own computed
+value. In 4.0.0 that write throws instead, so the failure is loud rather than a silent
+security gap.
+
+=== For parser authors
+
+* Document-derived key names (scraped from the file: HTML `<meta>` names, custom
+ document properties, CSV headers, ...) must go through a `KeyPrefix`'s typed factory
+ methods (`text`, `textBag`, `date`, `integer`, `real`, `bool`) rather than a raw
+ `String` key. See xref:developers/metadata-keys.adoc[Adding a Metadata Key].
+* `Metadata#set(String, String)` / `#add(String, String)` throw `IllegalArgumentException`
+ in 4.0.0 when the name is a reserved `tk:` (or legacy `X-TIKA:`) key. In 3.x this write
+ succeeded; there was no guard.
+* `Property`'s public factories (`Property.externalText(...)`, `internalText(...)`, etc.)
+ throw `IllegalArgumentException` at construction if given a reserved name -- a curated
+ `tk:` `Property` can only be built from Tika's own package-private factories.
+* The throw is a `RuntimeException` (`IllegalArgumentException`), not a `TikaException`.
+ A harness that narrowly catches `TikaException` around a parse will see this as a new,
+ uncaught failure mode when an unmigrated parser hits a crafted file.
+
+=== For integrators
+
+* `metadata.set("tk:...", value)` -- and any other map-style write to a reserved key --
+ throws `IllegalArgumentException` in 4.0.0.
+* A manual copy loop (`for (String n : src.names()) dest.set(n, src.get(n))`) both
+ collapses multi-valued keys to their last value *and* throws on every `tk:` key that
+ every parse output already contains (`tk:parsed-by`, `tk:content`, ...). Replace it with
+ `Metadata#putAll(Metadata)`, which preserves multi-values and copies reserved keys
+ through their trusted route.
+* `Metadata#setAll(Properties)` is removed; there is no replacement -- it wrote a raw
+ map directly into `Metadata`, bypassing both the limiter and the reserved-key guard.
+ Use `putAll(Metadata)` or individual `set`/`add` calls instead.
+* `/pipes` and `/async` `userMetadata` behavior is unchanged: a request's own `tk:`-named
+ entries still land in that request's output. This is deliberate (the deserializer uses
+ `Metadata`'s trusted reconstruction route, not the guarded `String` route) and is
+ bounded to the requester's own request; it is not affected by the guard changes above.
+
== Rationale
The namespacing of metadata keys provides several benefits:
diff --git a/docs/modules/ROOT/pages/migration-to-4x/migrating-to-4x.adoc b/docs/modules/ROOT/pages/migration-to-4x/migrating-to-4x.adoc
index a4d178eb59..2cb8df0c65 100644
--- a/docs/modules/ROOT/pages/migration-to-4x/migrating-to-4x.adoc
+++ b/docs/modules/ROOT/pages/migration-to-4x/migrating-to-4x.adoc
@@ -180,10 +180,11 @@ complete documentation of all available options.
== Metadata Key Changes
Tika 4.x prefixes all "user generated" metadata keys to prevent overwrites and improve
-namespace clarity.
+namespace clarity. Writing to a reserved `tk:` key by `String` name also changed: it
+silently succeeded in 3.x and now throws.
See xref:migration-to-4x/metadata-changes-4x.adoc[Metadata Changes in 4.x] for complete details, including
-a full table of changes and code migration examples.
+a full table of changes, the write-API/reserved-key-guard changes, and code migration examples.
== API Changes
@@ -244,6 +245,9 @@ for details.
== Deprecations and Removals
* `TikaConfig` -- replaced by `TikaLoader`
+* `Metadata#setAll(Properties)` -- raw map write that bypassed the reserved-key guard;
+ use `Metadata#putAll(Metadata)` instead (see
+ xref:migration-to-4x/metadata-changes-4x.adoc[Metadata Changes in 4.x])
* `CompositeExternalParser` -- external parsers now require explicit JSON configuration
* `ExternalParsersFactory` and XML-based external parser auto-discovery
* DOM-based OOXML extractors (`XWPFWordExtractorDecorator`, `XSLFPowerPointExtractorDecorator`)
diff --git a/tika-core/src/test/java/org/apache/tika/metadata/MetadataInternalKeyGuardTest.java b/tika-core/src/test/java/org/apache/tika/metadata/MetadataInternalKeyGuardTest.java
index 8603c5c27f..6bda921d26 100644
--- a/tika-core/src/test/java/org/apache/tika/metadata/MetadataInternalKeyGuardTest.java
+++ b/tika-core/src/test/java/org/apache/tika/metadata/MetadataInternalKeyGuardTest.java
@@ -139,4 +139,41 @@ public class MetadataInternalKeyGuardTest {
assertArrayEquals(new String[] {"p1", "p2"},
metadata.getValues(TikaCoreProperties.TIKA_PARSED_BY));
}
+
+ @Test
+ public void testReconstructNonReservedRoutesThroughStringPath() {
+ Metadata metadata = new Metadata();
+ metadata.reconstruct("my:customKey", "v1", false);
+ assertEquals("v1", metadata.get("my:customKey"));
+
+ metadata.reconstruct("my:customKey", "v2", true);
+ assertArrayEquals(new String[] {"v1", "v2"}, metadata.getValues("my:customKey"));
+ }
+
+ /**
+ * Design doc "Honest framing": {@code reconstruct} is a deliberately trusted route,
+ * not subject to the String-route guard -- for both a reserved name with a registered
+ * curated Property and one with none. Contrasts directly against the drop asserted by
+ * {@link #testStringWriteToInternalKeyIsDropped()} /
+ * {@link #testReconstructPreservesUnregisteredReservedKey()} on the same names, so a
+ * regression that made {@code reconstruct} start dropping (or the guard start
+ * exempting it) would be caught here either way.
+ */
+ @Test
+ public void testReconstructIsNotSubjectToReservedKeyDrop() {
+ Metadata metadata = new Metadata();
+
+ // registered curated Property
+ metadata.set(TikaCoreProperties.TIKA_CONTENT.getName(), "dropped-by-guard");
+ assertNull(metadata.get(TikaCoreProperties.TIKA_CONTENT));
+ metadata.reconstruct(TikaCoreProperties.TIKA_CONTENT.getName(), "lands-via-reconstruct", false);
+ assertEquals("lands-via-reconstruct", metadata.get(TikaCoreProperties.TIKA_CONTENT));
+
+ // reserved but unregistered
+ String unregistered = TikaCoreProperties.TIKA_META_PREFIX + "noSuchRegisteredProperty2";
+ metadata.set(unregistered, "dropped-by-guard");
+ assertNull(metadata.get(unregistered));
+ metadata.reconstruct(unregistered, "lands-via-reconstruct", false);
+ assertEquals("lands-via-reconstruct", metadata.get(unregistered));
+ }
}
diff --git a/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/serialization/UserMetadataReservedKeyStanceTest.java b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/serialization/UserMetadataReservedKeyStanceTest.java
new file mode 100644
index 0000000000..86c97179b2
--- /dev/null
+++ b/tika-pipes/tika-pipes-core/src/test/java/org/apache/tika/pipes/core/serialization/UserMetadataReservedKeyStanceTest.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.pipes.core.serialization;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+import java.io.IOException;
+import java.io.StringReader;
+
+import org.junit.jupiter.api.Test;
+
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.metadata.TikaCoreProperties;
+import org.apache.tika.pipes.api.FetchEmitTuple;
+
+/**
+ * Pins the accepted stance from the metadata-key-api design doc's "Honest framing"
+ * section: {@code FetchEmitTupleDeserializer} builds a tuple's userMetadata via
+ * {@link Metadata#reconstruct}, a deliberately trusted, request-reachable route -- so a
+ * requester CAN assert Tika's reserved {@code tk:} keys (e.g. {@code tk:content}) in
+ * their own request's userMetadata, and that value survives deserialization verbatim.
+ * This is accepted, not a bug: the requester only poisons their own request's output;
+ * deployment-level authn/authz on who may submit tuples is the actual trust boundary,
+ * not this deserializer (see design doc, "Honest framing"). If this test starts failing
+ * because the reserved key silently drops instead of landing, that is a stance change
+ * that must be re-approved, not "fixed."
+ */
+public class UserMetadataReservedKeyStanceTest {
+
+ @Test
+ public void userMetadataCanAssertReservedKeys_acceptedStance() throws IOException {
+ String json = """
+ {
+ "id": "id1",
+ "fetcher": "fs",
+ "fetchKey": "fetchKey1",
+ "metadata": {
+ "tk:content": "attacker-injected",
+ "tk:noSuchRegisteredProperty": "also-injected"
+ }
+ }
+ """;
+
+ FetchEmitTuple t = JsonFetchEmitTuple.fromJson(new StringReader(json));
+ Metadata metadata = t.getMetadata();
+
+ // registered curated Property: reconstruct routes through it, so it lands
+ assertEquals("attacker-injected", metadata.get(TikaCoreProperties.TIKA_CONTENT));
+ // unregistered reserved name: reconstruct's trusted-write branch, still lands
+ assertEquals("also-injected", metadata.get("tk:noSuchRegisteredProperty"));
+ }
+}