You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Lari Hotari <lh...@apache.org> on 2026/08/20 13:04:18 UTC
[NOTICE] Avro 1.12.1/1.12.2 security hardening breaks reflection-based schemas - impact on Pulsar and on client applications
Hi all,
While upgrading Avro from 1.12.0 to 1.12.2 (PR #24992, still in
review) I ran into a behaviour change that I think everyone working on
schemas, Functions or connectors should know about, because it also
affects user applications on upgrade and not just our own code.
Short version: as of Avro 1.12.2, Avro refuses to reflect over any
class that is not explicitly trusted. Pulsar's AVRO schemas are built
on Avro's reflection API, so without an allow-list every
Schema.AVRO(MyPojo.class) encode and decode throws SecurityException.
(Schema.JSON is unaffected - it goes through Jackson.)
WHAT CHANGED, AND IN WHICH RELEASE
----------------------------------
It is worth separating the two releases, since the change that causes
this landed in 1.12.2 rather than in 1.12.1.
Avro 1.12.1 (October 2025) shipped 4 security fixes. One of them
removed the default trusted packages, which used to be:
java.lang, java.math, java.io, java.net, org.apache.avro.reflect
But in 1.12.1 the allow-list was still consulted in exactly one place,
SpecificDatumReader.findStringClass. That means it only guarded
classes named by the "java-class" / "java-key-class" schema
properties. Record, enum and fixed types were still resolved with no
check at all. The class org.apache.avro.util.ClassSecurityValidator
does not exist in 1.12.1.
Avro 1.12.2 is where the scope changed. AVRO-4189 introduced
ClassSecurityValidator and moved the check into ClassUtils.forName, so
it now guards *every* reflective class resolution. What remains
trusted by default is 16 hardcoded JDK classes and no packages at all:
Boolean, Byte, Character, CharSequence, Double, Enum, Float, Integer,
Long, Number, Object, Short, String, Void, BigDecimal, BigInteger
That matters because SpecificData.getClass(Schema) resolves a class
for every RECORD, ENUM and FIXED schema, and
ReflectData.populateEncoderCache calls it via getCustomEncoding on
both ReflectDatumWriter.write and ReflectDatumReader.read. (Those
getCustomEncoding call sites are themselves new in 1.12.1, from
AVRO-4165.) So both the produce and the consume path go through the
check now, for every named type in the schema.
This is a significant tightening: in 1.12.1 the allow-list applied
only to classes named by java-class / java-key-class, and in 1.12.2 it
applies to every reflective class resolution. Plan for it as a
deliberate hardening that we adapt to.
Note that staying on 1.12.1 is not a workaround: AVRO-3940 in 1.12.1
introduced a StackOverflowError in ReflectData.getSchema for any POJO
with a recursive field (AVRO-4209), and that is only fixed in 1.12.2.
WHAT ACTUALLY BREAKS
--------------------
If you do nothing, on Avro 1.12.2:
* Producing or consuming with Schema.AVRO(MyPojo.class) fails with
java.lang.SecurityException: Forbidden com.example.MyPojo!
This class is not trusted to be included in Avro schemas.
wrapped in SchemaSerializationException. This hits both
directions, not just produce.
Schema.JSON is NOT affected: it derives an Avro schema for the
SchemaInfo but reads and writes through Jackson, so it never resolves
a class reflectively. Neither is Schema.AUTO_CONSUME over a record
schema, which stays generic. Schema.PROTOBUF is affected, but at
schema-construction time rather than on produce - avro-protobuf
resolves the generated message classes while building the schema.
* The failure is per named type, so it is not enough to trust the
top-level POJO. Every nested record and every enum reachable from it
is resolved separately, including types from other packages.
* A field's *declared* collection type is recorded by Avro as a
"java-class" property on the generated array/map schema and resolved
reflectively too. So a field declared List<Foo> needs java.util.List
trusted, a HashSet field needs java.util.HashSet, and so on.
* Nested classes have to be named with the binary '$' form. Avro
derives the schema name as com.example.Outer.Inner but resolves the
class by retrying with '$' separators, and the validator matches on
Class.getName(). An allow-list entry written with dots silently never
matches.
* @Stringable types are affected as before, but the previous
defaults are gone, so java.net.URI, java.net.URL and java.io.File now
need trusting explicitly. BigDecimal, BigInteger and Integer are still
covered by the hardcoded set.
* One non-obvious case: Schema.AUTO_CONSUME() against a topic whose
schema is Schema.JSON(SomeEnum.class). Nearly every POJO produces a
top-level Avro RECORD, and records are handled generically without
resolving any class. An enum is the exception - it produces a
top-level ENUM, and for a JSON-typed schema AutoConsumeSchema resolves
the class named in the *writer's* schema in order to rebuild a reader
schema for it. So the consumer needs the enum trusted, and the error
names it with the binary '$' form even though the schema spells it
with dots:
java.lang.SecurityException: Forbidden com.example.Outer$Colour!
Bare String or Integer POJOs are not affected - they produce plain
"string" / "int" schemas with nothing to resolve - and an AVRO-typed
enum schema takes a different branch that does not resolve either. It
is a narrow case, but not one anybody would predict from "Avro schemas
need trusted classes".
Two further 1.12.1 changes are worth flagging while we are here:
* AVRO-3230 enabled the fast reader by default. It reports malformed
data as AvroTypeException where the classic path raised
IndexOutOfBoundsException, so code that catches narrow exception types
around an Avro decode can now leak an Avro exception to callers.
* AVRO-4133 makes avro-protobuf emit a "default" attribute inside
generated enum schemas, which changes the schema JSON produced for a
protobuf class.
WHAT PULSAR DOES ABOUT IT (PR #24992)
-------------------------------------
Pulsar cannot know an application's classes, so the PR only takes
responsibility for what Pulsar itself serializes and for the code
Pulsar runs on the deployment's behalf.
* A new public class,
org.apache.pulsar.client.schema.AvroTrustedClasses, owns the trust and
installs it into Avro's global validator. It composes with whatever
validator is already installed, so Avro's own defaults, the
SERIALIZABLE_* system properties and any policy the application
installed itself all keep working. Its signatures use only Class,
String, ClassLoader and java.util.function.Predicate - deliberately no
Avro types, so it behaves the same in the shaded and unshaded clients.
* BrokerAvroTrustedClasses declares the broker's own Avro-serialized
types: transaction buffer snapshots, topic policy events on
__change_events, and the metadata events used by the metadata
synchronizer.
* FunctionAvroTrustedClasses trusts the class loader Pulsar created
for a deployed function or connector, so one using Schema.AVRO over
its own POJOs keeps working with no configuration, on all three
runtimes. Scoping to the class loader is deliberately narrower than a
wildcard: only the classes actually deployed become trusted, not
everything on the class path.
WHAT CLIENT APPLICATIONS HAVE TO DO
-----------------------------------
If your application uses Schema.AVRO with your own POJOs, declare them:
import org.apache.pulsar.client.schema.AvroTrustedClasses;
// Once, before the first producer or consumer.
AvroTrustedClasses.trust(Order.class);
trust(Class...) walks the schema Avro derives, so one call also covers
the nested records and enums it references - including ones in other
packages - and the declared collection and @Stringable types its
fields carry. That last part matters more than it sounds: a POJO with
a List field needs java.util.List trusted too, which is why "just
trust my model package" is usually not enough on its own. There is
also trustExactly(Class...) when you want only what you named,
trustClasses(String...) for classes you cannot reference at compile
time, trustPackages(String...), trustClassLoader(...) for plugin code
you load yourself, and trust(Predicate<Class<?>>) as an escape hatch.
A warning about Avro's own advice, because it is the first thing users
will read. The SecurityException message says to set
org.apache.avro.SERIALIZABLE_CLASSES / SERIALIZABLE_PACKAGES "or set
them via the API (see org.apache.avro.util.ClassSecurityValidator)".
For anyone on the shaded client - pulsar-client or pulsar-client-all,
which is the usual dependency - BOTH of those are wrong:
* The class is not on their class path. Shading relocates it to
org.apache.pulsar.shade.org.apache.avro.util.ClassSecurityValidator,
so the snippet does not compile. Adding org.apache.avro:avro to make
it compile is worse: it compiles, runs, configures a second copy of
the class and the SecurityException fires unchanged.
* The property name is relocated with it, so
-Dorg.apache.avro.SERIALIZABLE_PACKAGES is silently inert there.
So a shaded-client user who follows the exception gets a
byte-identical failure and concludes the upgrade is broken. That is
the main reason the PR adds AvroTrustedClasses rather than pointing at
Avro's API: we cannot change Avro's message, but we can give people
something that works either way.
The system properties do work if you use the right name for the artifact:
# pulsar-client / pulsar-client-all (shaded - the usual dependency)
-Dorg.apache.pulsar.shade.org.apache.avro.SERIALIZABLE_PACKAGES=com.example.model
# pulsar-client-original (unshaded), broker, proxy and other components
-Dorg.apache.avro.SERIALIZABLE_PACKAGES=com.example.model
SERIALIZABLE_CLASSES takes individual class names instead. Both are
read in a static initializer, so they have to be set before anything
in the JVM has touched Avro - a command-line -D always works, and
System.setProperty() works only if it runs first. Note that a package
on its own will not cover the java.util.List in the example above; you
would need SERIALIZABLE_CLASSES for that too, which is exactly the
bookkeeping trust(Class...) does for you.
It is also possible to turn the check off entirely, by trusting every
package - again with a different property name depending on the
artifact:
# pulsar-client / pulsar-client-all (shaded)
-Dorg.apache.pulsar.shade.org.apache.avro.SERIALIZABLE_PACKAGES=*
# pulsar-client-original (unshaded), broker, proxy and other components
-Dorg.apache.avro.SERIALIZABLE_PACKAGES=*
Avro treats "*" as trust-all, so this restores the pre-1.12.2
behaviour. It is there if the upgrade breaks an application and you
need it working again while you work out which classes to trust.
Whether to keep it that way is a policy decision for each deployment
to make on its own assessment.
This needs a release note, and I think it needs a documentation page
too. Feedback on the plan is welcome.
Thanks, Lari
Re: [NOTICE] Avro 1.12.1/1.12.2 security hardening breaks reflection-based schemas - impact on Pulsar and on client applications
Posted by Lari Hotari <lh...@apache.org>.
Hi all,
Following up on this. After discussing with Matteo, we decided the
upgrade should be far less invasive for Pulsar users, and that it was
worth spending the effort to find out whether that was possible. It
was, and the PR now meets that goal: existing client code does not
have to change.
The reason it is possible is that in Pulsar's case the application
already hands us the class. When you call Schema.AVRO(MyPojo.class),
that class - not some schema arriving over the wire - is the
authoritative description of what will be serialized. Pulsar derives
the Avro schema from it anyway, so it can walk that derived schema and
trust exactly the types Avro will resolve from it: the nested records
and enums it references, including ones in other packages, and the
collection and @Stringable types its fields declare. Nothing is
guessed and nothing is granted wholesale.
The paths where this happens are the ones where a Class is supplied:
* Schema.AVRO(Class)
* Schema.AVRO(SchemaDefinition) built with
SchemaDefinition.builder().withPojo(...)
* Schema.PROTOBUF(Class), where the message class is declared before
the schema is derived
Pulsar Functions and connectors are covered by the same mechanism,
because they build their schemas through those same calls, and so is
the broker for the types it writes to its own system topics.
Schema.JSON needs nothing declared either: it encodes and decodes
through Jackson rather than through Avro reflection.
What is deliberately not covered is a class named only by a schema
that arrived from the schema registry, since that is exactly the case
the Avro allow-list exists to constrain. Schema.AUTO_CONSUME() is
unaffected in the ordinary case: a record schema is decoded
generically into a GenericRecord and no class is resolved at all,
nested records included.
The one gap is a topic whose schema is not a record at the top level,
which is not typical of existing Pulsar usage - a producer has to have
created the topic with something like Schema.AVRO(SomeEnum.class). For
those topics Pulsar decodes with Avro's reflection-based reader, which
loads a Java class for every record, enum and fixed type the schema
names. Putting the type inside an array or a map does not avoid this;
the class is loaded as soon as there is an element to read. Only
schemas that name no such type at all, a string or an array of strings
for instance, are unaffected. An application consuming such a topic
declares the class itself through
org.apache.pulsar.client.schema.AvroTrustedClasses, which is also
there for anything else that needs widening. Since topics of that
shape are unusual to begin with, the risk of this breaking an existing
Pulsar application is minor.
Older Avro versions are handled too.
org.apache.avro.util.ClassSecurityValidator only exists from Avro
1.12.2, and an application on pulsar-client-original (the unshaded
client) can pin an older Avro. Where the class is absent there is
nothing to enforce, so declaring trust is a no-op rather than a
NoClassDefFoundError, and this is covered by a test.
Reviewers: please see https://github.com/apache/pulsar/pull/24992 for
the details.
Thanks,
Lari
On Thu, 20 Aug 2026 at 16:04, Lari Hotari <lh...@apache.org> wrote:
>
> Hi all,
>
> While upgrading Avro from 1.12.0 to 1.12.2 (PR #24992, still in
> review) I ran into a behaviour change that I think everyone working on
> schemas, Functions or connectors should know about, because it also
> affects user applications on upgrade and not just our own code.
>
> Short version: as of Avro 1.12.2, Avro refuses to reflect over any
> class that is not explicitly trusted. Pulsar's AVRO schemas are built
> on Avro's reflection API, so without an allow-list every
> Schema.AVRO(MyPojo.class) encode and decode throws SecurityException.
> (Schema.JSON is unaffected - it goes through Jackson.)
>
>
> WHAT CHANGED, AND IN WHICH RELEASE
> ----------------------------------
>
> It is worth separating the two releases, since the change that causes
> this landed in 1.12.2 rather than in 1.12.1.
>
> Avro 1.12.1 (October 2025) shipped 4 security fixes. One of them
> removed the default trusted packages, which used to be:
>
> java.lang, java.math, java.io, java.net, org.apache.avro.reflect
>
> But in 1.12.1 the allow-list was still consulted in exactly one place,
> SpecificDatumReader.findStringClass. That means it only guarded
> classes named by the "java-class" / "java-key-class" schema
> properties. Record, enum and fixed types were still resolved with no
> check at all. The class org.apache.avro.util.ClassSecurityValidator
> does not exist in 1.12.1.
>
> Avro 1.12.2 is where the scope changed. AVRO-4189 introduced
> ClassSecurityValidator and moved the check into ClassUtils.forName, so
> it now guards *every* reflective class resolution. What remains
> trusted by default is 16 hardcoded JDK classes and no packages at all:
>
> Boolean, Byte, Character, CharSequence, Double, Enum, Float, Integer,
> Long, Number, Object, Short, String, Void, BigDecimal, BigInteger
>
> That matters because SpecificData.getClass(Schema) resolves a class
> for every RECORD, ENUM and FIXED schema, and
> ReflectData.populateEncoderCache calls it via getCustomEncoding on
> both ReflectDatumWriter.write and ReflectDatumReader.read. (Those
> getCustomEncoding call sites are themselves new in 1.12.1, from
> AVRO-4165.) So both the produce and the consume path go through the
> check now, for every named type in the schema.
>
> This is a significant tightening: in 1.12.1 the allow-list applied
> only to classes named by java-class / java-key-class, and in 1.12.2 it
> applies to every reflective class resolution. Plan for it as a
> deliberate hardening that we adapt to.
>
> Note that staying on 1.12.1 is not a workaround: AVRO-3940 in 1.12.1
> introduced a StackOverflowError in ReflectData.getSchema for any POJO
> with a recursive field (AVRO-4209), and that is only fixed in 1.12.2.
>
>
> WHAT ACTUALLY BREAKS
> --------------------
>
> If you do nothing, on Avro 1.12.2:
>
> * Producing or consuming with Schema.AVRO(MyPojo.class) fails with
>
> java.lang.SecurityException: Forbidden com.example.MyPojo!
> This class is not trusted to be included in Avro schemas.
>
> wrapped in SchemaSerializationException. This hits both
> directions, not just produce.
>
> Schema.JSON is NOT affected: it derives an Avro schema for the
> SchemaInfo but reads and writes through Jackson, so it never resolves
> a class reflectively. Neither is Schema.AUTO_CONSUME over a record
> schema, which stays generic. Schema.PROTOBUF is affected, but at
> schema-construction time rather than on produce - avro-protobuf
> resolves the generated message classes while building the schema.
>
> * The failure is per named type, so it is not enough to trust the
> top-level POJO. Every nested record and every enum reachable from it
> is resolved separately, including types from other packages.
>
> * A field's *declared* collection type is recorded by Avro as a
> "java-class" property on the generated array/map schema and resolved
> reflectively too. So a field declared List<Foo> needs java.util.List
> trusted, a HashSet field needs java.util.HashSet, and so on.
>
> * Nested classes have to be named with the binary '$' form. Avro
> derives the schema name as com.example.Outer.Inner but resolves the
> class by retrying with '$' separators, and the validator matches on
> Class.getName(). An allow-list entry written with dots silently never
> matches.
>
> * @Stringable types are affected as before, but the previous
> defaults are gone, so java.net.URI, java.net.URL and java.io.File now
> need trusting explicitly. BigDecimal, BigInteger and Integer are still
> covered by the hardcoded set.
>
> * One non-obvious case: Schema.AUTO_CONSUME() against a topic whose
> schema is Schema.JSON(SomeEnum.class). Nearly every POJO produces a
> top-level Avro RECORD, and records are handled generically without
> resolving any class. An enum is the exception - it produces a
> top-level ENUM, and for a JSON-typed schema AutoConsumeSchema resolves
> the class named in the *writer's* schema in order to rebuild a reader
> schema for it. So the consumer needs the enum trusted, and the error
> names it with the binary '$' form even though the schema spells it
> with dots:
>
> java.lang.SecurityException: Forbidden com.example.Outer$Colour!
>
> Bare String or Integer POJOs are not affected - they produce plain
> "string" / "int" schemas with nothing to resolve - and an AVRO-typed
> enum schema takes a different branch that does not resolve either. It
> is a narrow case, but not one anybody would predict from "Avro schemas
> need trusted classes".
>
> Two further 1.12.1 changes are worth flagging while we are here:
>
> * AVRO-3230 enabled the fast reader by default. It reports malformed
> data as AvroTypeException where the classic path raised
> IndexOutOfBoundsException, so code that catches narrow exception types
> around an Avro decode can now leak an Avro exception to callers.
>
> * AVRO-4133 makes avro-protobuf emit a "default" attribute inside
> generated enum schemas, which changes the schema JSON produced for a
> protobuf class.
>
>
> WHAT PULSAR DOES ABOUT IT (PR #24992)
> -------------------------------------
>
> Pulsar cannot know an application's classes, so the PR only takes
> responsibility for what Pulsar itself serializes and for the code
> Pulsar runs on the deployment's behalf.
>
> * A new public class,
> org.apache.pulsar.client.schema.AvroTrustedClasses, owns the trust and
> installs it into Avro's global validator. It composes with whatever
> validator is already installed, so Avro's own defaults, the
> SERIALIZABLE_* system properties and any policy the application
> installed itself all keep working. Its signatures use only Class,
> String, ClassLoader and java.util.function.Predicate - deliberately no
> Avro types, so it behaves the same in the shaded and unshaded clients.
>
> * BrokerAvroTrustedClasses declares the broker's own Avro-serialized
> types: transaction buffer snapshots, topic policy events on
> __change_events, and the metadata events used by the metadata
> synchronizer.
>
> * FunctionAvroTrustedClasses trusts the class loader Pulsar created
> for a deployed function or connector, so one using Schema.AVRO over
> its own POJOs keeps working with no configuration, on all three
> runtimes. Scoping to the class loader is deliberately narrower than a
> wildcard: only the classes actually deployed become trusted, not
> everything on the class path.
>
>
> WHAT CLIENT APPLICATIONS HAVE TO DO
> -----------------------------------
>
> If your application uses Schema.AVRO with your own POJOs, declare them:
>
> import org.apache.pulsar.client.schema.AvroTrustedClasses;
>
> // Once, before the first producer or consumer.
> AvroTrustedClasses.trust(Order.class);
>
> trust(Class...) walks the schema Avro derives, so one call also covers
> the nested records and enums it references - including ones in other
> packages - and the declared collection and @Stringable types its
> fields carry. That last part matters more than it sounds: a POJO with
> a List field needs java.util.List trusted too, which is why "just
> trust my model package" is usually not enough on its own. There is
> also trustExactly(Class...) when you want only what you named,
> trustClasses(String...) for classes you cannot reference at compile
> time, trustPackages(String...), trustClassLoader(...) for plugin code
> you load yourself, and trust(Predicate<Class<?>>) as an escape hatch.
>
> A warning about Avro's own advice, because it is the first thing users
> will read. The SecurityException message says to set
> org.apache.avro.SERIALIZABLE_CLASSES / SERIALIZABLE_PACKAGES "or set
> them via the API (see org.apache.avro.util.ClassSecurityValidator)".
> For anyone on the shaded client - pulsar-client or pulsar-client-all,
> which is the usual dependency - BOTH of those are wrong:
>
> * The class is not on their class path. Shading relocates it to
> org.apache.pulsar.shade.org.apache.avro.util.ClassSecurityValidator,
> so the snippet does not compile. Adding org.apache.avro:avro to make
> it compile is worse: it compiles, runs, configures a second copy of
> the class and the SecurityException fires unchanged.
>
> * The property name is relocated with it, so
> -Dorg.apache.avro.SERIALIZABLE_PACKAGES is silently inert there.
>
> So a shaded-client user who follows the exception gets a
> byte-identical failure and concludes the upgrade is broken. That is
> the main reason the PR adds AvroTrustedClasses rather than pointing at
> Avro's API: we cannot change Avro's message, but we can give people
> something that works either way.
>
> The system properties do work if you use the right name for the artifact:
>
> # pulsar-client / pulsar-client-all (shaded - the usual dependency)
> -Dorg.apache.pulsar.shade.org.apache.avro.SERIALIZABLE_PACKAGES=com.example.model
>
> # pulsar-client-original (unshaded), broker, proxy and other components
> -Dorg.apache.avro.SERIALIZABLE_PACKAGES=com.example.model
>
> SERIALIZABLE_CLASSES takes individual class names instead. Both are
> read in a static initializer, so they have to be set before anything
> in the JVM has touched Avro - a command-line -D always works, and
> System.setProperty() works only if it runs first. Note that a package
> on its own will not cover the java.util.List in the example above; you
> would need SERIALIZABLE_CLASSES for that too, which is exactly the
> bookkeeping trust(Class...) does for you.
>
> It is also possible to turn the check off entirely, by trusting every
> package - again with a different property name depending on the
> artifact:
>
> # pulsar-client / pulsar-client-all (shaded)
> -Dorg.apache.pulsar.shade.org.apache.avro.SERIALIZABLE_PACKAGES=*
>
> # pulsar-client-original (unshaded), broker, proxy and other components
> -Dorg.apache.avro.SERIALIZABLE_PACKAGES=*
>
> Avro treats "*" as trust-all, so this restores the pre-1.12.2
> behaviour. It is there if the upgrade breaks an application and you
> need it working again while you work out which classes to trust.
> Whether to keep it that way is a policy decision for each deployment
> to make on its own assessment.
>
> This needs a release note, and I think it needs a documentation page
> too. Feedback on the plan is welcome.
>
> Thanks, Lari