Flink in ECS fails to find shaded ContainerCredentialsProvider Flink in ECS fails to find shaded ContainerCredentialsProvider hadoop hadoop

Flink in ECS fails to find shaded ContainerCredentialsProvider


hadoop amazon-s3 apache-flink flink-streaming

The issue was resolved in one of the versions after.

I ran it on Flink 1.9.0 cluster with the following line:

RUN echo "fs.s3a.aws.credentials.provider: com.amazonaws.auth.ContainerCredentialsProvider" >> "$FLINK_CONF_DIR/flink-conf.yaml"

and the class is found and works.

You can see in:https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-hadoop/src/main/java/org/apache/flink/fs/s3hadoop/S3FileSystemFactory.java

that the FLINK_SHADING_PREFIX is now correct

Recent Posts
How can I color dots in a xy scatterplot according to column value?
How to update a claim in ASP.NET Identity?
What does {0} mean when initializing an object?
Accessing members of items in a JSONArray with Java
How to log SQL statements in Spring Boot?
Powershell Get-WebSite name parameter is ignored
How to detect scroll to bottom of html element
Java synchronized method
How to test controllers with CodeIgniter?
Detect Visual Composer
Matplotlib: Specify format of floats for tick labels
Rails join a list of strings with commas and "and" before the last