You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Stanislav Lukyanov <st...@gmail.com> on 2019/01/10 16:25:54 UTC

RE: Amazon S3 Based Discovery NOT USING BasicAWSCredentials

Hi,

Were you able to solve this?

It seems that your config is actually fine… The feature was added by https://issues.apache.org/jira/browse/IGNITE-4530.

Does it work if you replace `ref` with just a value?
Like 
<property name="awsCredentialsProvider">
    <bean class="com.amazonaws.auth.InstanceProfileCredentialsProvider"/>
</property>

Stan

From: Max Barrios
Sent: 12 декабря 2018 г. 23:51
To: user@ignite.apache.org
Subject: Amazon S3 Based Discovery NOT USING BasicAWSCredentials

I am running Apache Ignite 2.6.0 in AWS and am using S3 Based Discovery,

However, I DO NOT want to embed AWS Access or Secret Keys in my ignite.xml

I have AWS EC2 Instance Metadata Service for my instances so that the creds can be loaded from there. 

However, there's no guidance or documentation on how to do this. Is this even supported?

For example, I want to do this:
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  ...
  <property name="discoverySpi">
    <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
      <property name="ipFinder">
        <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">
          <property name="awsCredentialsProvider" ref="aws.creds"/>
          <property name="bucketName" value="YOUR_BUCKET_NAME"/>
        </bean>
      </property>
    </bean>
  </property>
</bean>

<!-- AWS credentials. Provide your access key ID and secret access key. -->
<bean id="aws.creds" class="com.amazonaws.auth.InstanceProfileCredentialsProvider">
</bean>

But I get this exception when I try the above:

Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'awsCredentialsProvider' of bean class [org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder]: Bean property 'awsCredentialsProvider' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?

If using an AWS Credentials Provider *is* supported, where are the bean properties for documented, so I can see what I may be doing wrong? Are there any working examples for anything other than BasicAWSCCredentials?

Please help. 

Max

RE: Amazon S3 Based Discovery NOT USING BasicAWSCredentials

Posted by Stanislav Lukyanov <st...@gmail.com>.
awsCredentialsProvider is just a property of the  TcpDiscoveryS3IpFinder.
If the class is loaded then the property should also be there.

I’ll try this as well and let you know if that works for me.

Stan

From: Max Barrios
Sent: 11 января 2019 г. 4:47
To: user@ignite.apache.org
Subject: Re: Amazon S3 Based Discovery NOT USING BasicAWSCredentials

I’m still seeing this error even when passing a value via ‘ref’. 

Looks like there’s some ignite lib not getting loaded that can resolve the awsCredentials property, when running in a spark cluster.

Is there any comprehensive guidance for running a spark app using ignite in a spark cluster? 
Sent from my iPhone

On Jan 10, 2019, at 08:25, Stanislav Lukyanov <st...@gmail.com> wrote:
Hi,
 
Were you able to solve this?
 
It seems that your config is actually fine… The feature was added by https://issues.apache.org/jira/browse/IGNITE-4530.
 
Does it work if you replace `ref` with just a value?
Like 
<property name="awsCredentialsProvider">
    <bean class="com.amazonaws.auth.InstanceProfileCredentialsProvider"/>
</property>
 
Stan
 
From: Max Barrios
Sent: 12 декабря 2018 г. 23:51
To: user@ignite.apache.org
Subject: Amazon S3 Based Discovery NOT USING BasicAWSCredentials
 
I am running Apache Ignite 2.6.0 in AWS and am using S3 Based Discovery,
 
However, I DO NOT want to embed AWS Access or Secret Keys in my ignite.xml
 
I have AWS EC2 Instance Metadata Service for my instances so that the creds can be loaded from there. 
 
However, there's no guidance or documentation on how to do this. Is this even supported?
 
For example, I want to do this:
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  ...
  <property name="discoverySpi">
    <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
      <property name="ipFinder">
        <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">
          <property name="awsCredentialsProvider" ref="aws.creds"/>
          <property name="bucketName" value="YOUR_BUCKET_NAME"/>
        </bean>
      </property>
    </bean>
  </property>
</bean>
 
<!-- AWS credentials. Provide your access key ID and secret access key. -->
<bean id="aws.creds" class="com.amazonaws.auth.InstanceProfileCredentialsProvider">
</bean>
 
But I get this exception when I try the above:
 
Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'awsCredentialsProvider' of bean class [org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder]: Bean property 'awsCredentialsProvider' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
 
If using an AWS Credentials Provider *is* supported, where are the bean properties for documented, so I can see what I may be doing wrong? Are there any working examples for anything other than BasicAWSCCredentials?
 
Please help. 
 
Max
 

Re: Amazon S3 Based Discovery NOT USING BasicAWSCredentials

Posted by Max Barrios <sp...@gmail.com>.
I’m still seeing this error even when passing a value via ‘ref’. 

Looks like there’s some ignite lib not getting loaded that can resolve the awsCredentials property, when running in a spark cluster.

Is there any comprehensive guidance for running a spark app using ignite in a spark cluster? 

Sent from my iPhone

> On Jan 10, 2019, at 08:25, Stanislav Lukyanov <st...@gmail.com> wrote:
> 
> Hi,
>  
> Were you able to solve this?
>  
> It seems that your config is actually fine… The feature was added by https://issues.apache.org/jira/browse/IGNITE-4530.
>  
> Does it work if you replace `ref` with just a value?
> Like
> <property name="awsCredentialsProvider">
>     <bean class="com.amazonaws.auth.InstanceProfileCredentialsProvider"/>
> </property>
>  
> Stan
>  
> From: Max Barrios
> Sent: 12 декабря 2018 г. 23:51
> To: user@ignite.apache.org
> Subject: Amazon S3 Based Discovery NOT USING BasicAWSCredentials
>  
> I am running Apache Ignite 2.6.0 in AWS and am using S3 Based Discovery,
>  
> However, I DO NOT want to embed AWS Access or Secret Keys in my ignite.xml
>  
> I have AWS EC2 Instance Metadata Service for my instances so that the creds can be loaded from there. 
>  
> However, there's no guidance or documentation on how to do this. Is this even supported?
>  
> For example, I want to do this:
> <bean class="org.apache.ignite.configuration.IgniteConfiguration">
>   ...
>   <property name="discoverySpi">
>     <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>       <property name="ipFinder">
>         <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">
>           <property name="awsCredentialsProvider" ref="aws.creds"/>
>           <property name="bucketName" value="YOUR_BUCKET_NAME"/>
>         </bean>
>       </property>
>     </bean>
>   </property>
> </bean>
>  
> <!-- AWS credentials. Provide your access key ID and secret access key. -->
> <bean id="aws.creds" class="com.amazonaws.auth.InstanceProfileCredentialsProvider">
> </bean>
>  
> But I get this exception when I try the above:
>  
> Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'awsCredentialsProvider' of bean class [org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder]: Bean property 'awsCredentialsProvider' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
>  
> If using an AWS Credentials Provider *is* supported, where are the bean properties for documented, so I can see what I may be doing wrong? Are there any working examples for anything other than BasicAWSCCredentials?
>  
> Please help. 
>  
> Max
>

Re: Amazon S3 Based Discovery NOT USING BasicAWSCredentials

Posted by Max Barrios <sp...@gmail.com>.
I’m still seeing this problem

I’ll try the suggested approach 

Sent from my iPhone

> On Jan 10, 2019, at 08:25, Stanislav Lukyanov <st...@gmail.com> wrote:
> 
> Hi,
>  
> Were you able to solve this?
>  
> It seems that your config is actually fine… The feature was added by https://issues.apache.org/jira/browse/IGNITE-4530.
>  
> Does it work if you replace `ref` with just a value?
> Like
> <property name="awsCredentialsProvider">
>     <bean class="com.amazonaws.auth.InstanceProfileCredentialsProvider"/>
> </property>
>  
> Stan
>  
> From: Max Barrios
> Sent: 12 декабря 2018 г. 23:51
> To: user@ignite.apache.org
> Subject: Amazon S3 Based Discovery NOT USING BasicAWSCredentials
>  
> I am running Apache Ignite 2.6.0 in AWS and am using S3 Based Discovery,
>  
> However, I DO NOT want to embed AWS Access or Secret Keys in my ignite.xml
>  
> I have AWS EC2 Instance Metadata Service for my instances so that the creds can be loaded from there. 
>  
> However, there's no guidance or documentation on how to do this. Is this even supported?
>  
> For example, I want to do this:
> <bean class="org.apache.ignite.configuration.IgniteConfiguration">
>   ...
>   <property name="discoverySpi">
>     <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>       <property name="ipFinder">
>         <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">
>           <property name="awsCredentialsProvider" ref="aws.creds"/>
>           <property name="bucketName" value="YOUR_BUCKET_NAME"/>
>         </bean>
>       </property>
>     </bean>
>   </property>
> </bean>
>  
> <!-- AWS credentials. Provide your access key ID and secret access key. -->
> <bean id="aws.creds" class="com.amazonaws.auth.InstanceProfileCredentialsProvider">
> </bean>
>  
> But I get this exception when I try the above:
>  
> Error setting property values; nested exception is org.springframework.beans.NotWritablePropertyException: Invalid property 'awsCredentialsProvider' of bean class [org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder]: Bean property 'awsCredentialsProvider' is not writable or has an invalid setter method. Does the parameter type of the setter match the return type of the getter?
>  
> If using an AWS Credentials Provider *is* supported, where are the bean properties for documented, so I can see what I may be doing wrong? Are there any working examples for anything other than BasicAWSCCredentials?
>  
> Please help. 
>  
> Max
>