You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by vishalovercome <vi...@moengage.com> on 2020/05/20 16:46:39 UTC
Issues using StreamingFileSink (Cannot locate configuration: tried
hadoop-metrics2-s3a-file-system.properties, hadoop-metrics2.properties)
I've written a Flink job to stream updates to S3 using StreamingFileSink. I
used the presto plugin for checkpointing and the hadoop plugin for writing
part files to S3. I was going through my application logs and found these
messages which I couldn't understand and would like to know if there's some
step I might have missed.
Even if these messages are harmless, I would like to understand what these
mean.
2020-05-19 19:23:33,277 INFO
org.apache.flink.fs.shaded.hadoop3.org.apache.commons.beanutils.FluentPropertyBeanIntrospector
- Error when creating PropertyDescriptor for public final void
org.apache.flink.fs.shaded.hadoop3.org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)!
Ignoring this property.
*2020-05-19 19:23:33,310 WARN
org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsConfig
- Cannot locate configuration: tried
hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties*
2020-05-19 19:23:33,397 INFO
org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsSystemImpl
- Scheduled Metric snapshot period at 10 second(s).
2020-05-19 19:23:33,397 INFO
org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsSystemImpl
- s3a-file-system metrics system started
2020-05-19 19:23:34,833 INFO
org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.conf.Configuration.deprecation
- fs.s3a.server-side-encryption-key is deprecated. Instead, use
fs.s3a.server-side-encryption.key
--
Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
Re: Issues using StreamingFileSink (Cannot locate configuration:
tried hadoop-metrics2-s3a-file-system.properties, hadoop-metrics2.properties)
Posted by Till Rohrmann <tr...@apache.org>.
Hi,
for the first message I think this might be related to BEANUTILS-477 [1]
and should not be a problem.
The other messages look to me like the metric system initialization of the
Hadoop components. I don't think that there is something wrong with them.
[1] https://issues.apache.org/jira/browse/BEANUTILS-477
Cheers,
Till
On Wed, May 20, 2020 at 6:46 PM vishalovercome <vi...@moengage.com> wrote:
> I've written a Flink job to stream updates to S3 using StreamingFileSink. I
> used the presto plugin for checkpointing and the hadoop plugin for writing
> part files to S3. I was going through my application logs and found these
> messages which I couldn't understand and would like to know if there's some
> step I might have missed.
>
> Even if these messages are harmless, I would like to understand what these
> mean.
>
> 2020-05-19 19:23:33,277 INFO
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.commons.beanutils.FluentPropertyBeanIntrospector
> - Error when creating PropertyDescriptor for public final void
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)!
> Ignoring this property.
>
> *2020-05-19 19:23:33,310 WARN
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsConfig
> - Cannot locate configuration: tried
> hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties*
>
> 2020-05-19 19:23:33,397 INFO
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsSystemImpl
> - Scheduled Metric snapshot period at 10 second(s).
>
> 2020-05-19 19:23:33,397 INFO
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsSystemImpl
> - s3a-file-system metrics system started
>
> 2020-05-19 19:23:34,833 INFO
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.conf.Configuration.deprecation
> - fs.s3a.server-side-encryption-key is deprecated. Instead, use
> fs.s3a.server-side-encryption.key
>
>
>
> --
> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>