You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Karan Makhija (Jira)" <ji...@apache.org> on 2025/11/25 09:34:00 UTC
[jira] [Created] (FLINK-38725) Flink avro gives Security Exception with flink-sql-avro.jar
Karan Makhija created FLINK-38725:
-------------------------------------
Summary: Flink avro gives Security Exception with flink-sql-avro.jar
Key: FLINK-38725
URL: https://issues.apache.org/jira/browse/FLINK-38725
Project: Flink
Issue Type: Bug
Affects Versions: 1.20.0
Reporter: Karan Makhija
Avro schema which with BigDecimal gives the following exception:
Caused by: java.lang.SecurityException: Forbidden class java.math.BigDecimal! This class is not trusted to be included in Avro schema using java-class. Please set org.apache.avro.SERIALIZABLE_PACKAGES system property with the packages you trust. at org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.checkSecurity([SpecificDatumReader.java:145|http://specificdatumreader.java:145/]) at org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.getPropAsClass([SpecificDatumReader.java:119|http://specificdatumreader.java:119/]) at org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.findStringClass([SpecificDatumReader.java:102|http://specificdatumreader.java:102/]) at org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader$ReaderCache.lambda$getStringClass$1([GenericDatumReader.java:567|http://genericdatumreader.java:567/]) at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent([ConcurrentHashMap.java:1705|http://concurrenthashmap.java:1705/]) at org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader$ReaderCache.getStringClass([GenericDatumReader.java:567|http://genericdatumreader.java:567/]) at org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readString([GenericDatumReader.java:455|http://genericdatumreader.java:455/]) at org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readWithoutConversion([GenericDatumReader.java:192|http://genericdatumreader.java:192/]) at org.apache.flink.avro.shaded.org.apache.avro.reflect.ReflectDatumReader.readField([ReflectDatumReader.java:301|http://reflectdatumreader.java:301/]) at org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readRecord([GenericDatumReader.java:248|http://genericdatumreader.java:248/]) at org.apache.flink.avro.shaded.org.apache.avro.specific.SpecificDatumReader.readRecord([SpecificDatumReader.java:168|http://specificdatumreader.java:168/]) at org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.readWithoutConversion([GenericDatumReader.java:180|http://genericdatumreader.java:180/]) at [org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.read|http://org.apache.flink.avro.shaded.org.apache.avro.generic.genericdatumreader.read/]([GenericDatumReader.java:161|http://genericdatumreader.java:161/]) at [org.apache.flink.avro.shaded.org.apache.avro.generic.GenericDatumReader.read|http://org.apache.flink.avro.shaded.org.apache.avro.generic.genericdatumreader.read/]([GenericDatumReader.java:154|http://genericdatumreader.java:154/]) at [org.apache.flink.avro.shaded.org.apache.avro.file.DataFileStream.next|http://org.apache.flink.avro.shaded.org.apache.avro.file.datafilestream.next/]([DataFileStream.java:263|http://datafilestream.java:263/]) at org.apache.flink.formats.avro.AvroInputFormat.nextRecord([AvroInputFormat.java:170|http://avroinputformat.java:170/]) at org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.readAndCollectRecord([ContinuousFileReaderOperator.java:390|http://continuousfilereaderoperator.java:390/]) at org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.processRecord([ContinuousFileReaderOperator.java:352|http://continuousfilereaderoperator.java:352/]) at org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator.lambda$new$0([ContinuousFileReaderOperator.java:240|http://continuousfilereaderoperator.java:240/]) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing([StreamTaskActionExecutor.java:50|http://streamtaskactionexecutor.java:50/]) at [org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run|http://org.apache.flink.streaming.runtime.tasks.mailbox.mail.run/]([Mail.java:101|http://mail.java:101/]) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail([MailboxProcessor.java:414|http://mailboxprocessor.java:414/]) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsNonBlocking([MailboxProcessor.java:399|http://mailboxprocessor.java:399/]) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail([MailboxProcessor.java:361|http://mailboxprocessor.java:361/]) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop([MailboxProcessor.java:229|http://mailboxprocessor.java:229/]) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop([StreamTask.java:973|http://streamtask.java:973/]) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke([StreamTask.java:917|http://streamtask.java:917/]) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring([Task.java:970|http://task.java:970/]) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke([Task.java:949|http://task.java:949/]) at org.apache.flink.runtime.taskmanager.Task.doRun([Task.java:763|http://task.java:763/]) at [org.apache.flink.runtime.taskmanager.Task.run|http://org.apache.flink.runtime.taskmanager.task.run/]([Task.java:575|http://task.java:575/]) at java.base/java.lang.Thread.run([Thread.java:829|http://thread.java:829/])
The cluster configuration included a flink-conf property: "env.java.opts": "-Dorg.apache.flink.avro.shaded.org.apache.avro.SERIALIZABLE_PACKAGESS=java.math -Dorg.apache.avro.SERIALIZABLE_PACKAGES=java.math". The same is working after I remove the relocations from the flink-sql-avro.jar.
{{[hadoop@ip-172-31-16-8 ~]$ flink run flink-job.jar --source s3://<S3-BUCKET>/data/
2025-11-20 11:12:32,971 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /var/lib/flink/yarn/.yarn-properties-hadoop.
2025-11-20 11:12:32,971 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /var/lib/flink/yarn/.yarn-properties-hadoop. 2025-11-20 11:12:33,665 INFO org.apache.hadoop.metrics2.impl.MetricsConfig [] - Loaded properties from hadoop-metrics2.properties
2025-11-20 11:12:33,678 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl [] - Scheduled Metric snapshot period at 300 second(s).
2025-11-20 11:12:33,678 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl [] - s3a-file-system metrics system started
Reading data source 1: }}{{s3://<S3-BUCKET>/data/}}{{{} Skipping filter step Adding execution job 2025-11-20 11:12:34,847 INFO org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider [] - Connecting to ResourceManager at ip-172-31-16-8.ec2.internal/172.31.16.8:8032 2025-11-20 11:12:34,932 INFO org.apache.hadoop.yarn.client.AHSProxy [] - Connecting to Application History server at ip-172-31-16-8.ec2.internal/172.31.16.8:10200 2025-11-20 11:12:34,938 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar 2025-11-20 11:12:35,026 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web Interface ip-172-31-30-122.ec2.internal:41139 of application 'application_1763613799717_0008'. Job has been submitted with JobID 1bad2e49ee0da8894a65a98133b894ce Program execution finished Job with JobID 1bad2e49ee0da8894a65a98133b894ce has finished. Job Runtime: 21012 ms Total input records: 0 Total output records: 0 timestamp: 2025-11-20T11:12:58.596487112Z inputTotal: 0 outputTotal: 0 parameters: {source={}}}{{{}s3://<S3-BUCKET>/data/{}}}{{{}}{}}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)