You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by theizer <ti...@nrccua.org> on 2018/03/26 16:28:57 UTC
Ingite deploy in AWS
I am trying to deploy an Apache Ignite cluster in our AWS VPC. I have done
the following:
• Created an S3 Bucket to store the ignite-config.xml file
o Uploaded the config file to the bucket.
Config file contents (just based on example in github):
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:util="http://www.springframework.org/schema/util"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<bean id="grid.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="cacheConfiguration">
<list>
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="IgniteCache"/>
<property name="cacheMode" value="PARTITIONED"/>
<property name="atomicWriteOrderMode" value="PRIMARY"/>
<property name="writeSynchronizationMode" value="PRIMARY_SYNC"/>
<property name="evictionPolicy">
<bean
class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
<property name="maxSize" value="100000000"/>
</bean>
</property>
<property name="swapEnabled" value="false"/>
<property name="atomicityMode" value="ATOMIC" />
<property name="backups" value="1" />
</bean>
</list>
</property>
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">
<property name="awsCredentials" ref="aws.creds"/>
<property name="bucketName"
value="S3BucketNAME-ignite-discovery"/>
</bean>
</property>
</bean>
</property>
<property name="communicationSpi">
<bean
class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
<property name="slowClientQueueLimit" value="1000"/>
</bean>
</property>
</bean>
<bean id="aws.creds" class="com.amazonaws.auth.BasicAWSCredentials">
<constructor-arg value="ABCDEFGHIJKLMNOPRST" />
<constructor-arg value="A1B2C3D4E5F6G7H8I9J1A1B2C3D4E5F6G7H8" />
</bean>
</beans>
• Created an S3 bucket for S3 Discovery
• Created an IAM role for Ignite-S3-Access with the AmazoneS3FullAccess
policy attached and ec2.amazonaws.com as a trusted entity
• Launched an EC2 instance from the Apache Ignite AMI for the us-east-1
region
o In my VPC private subnet
o With IAM role created above for S3 Access
o User Data:
CONFIG_URI=https://s3.amazonaws.com/S3BucketName-ignite-config/ignite-config.xml
When I log into the instance, the docker container is NOT running.
I can verify that the IAM role for S3 is working by typing aws s3 ls and
getting a list of all s3 buckets.
If I try to manually start the docker container with the command: sudo
docker run -it --net=host -e
"CONFIG_URI=https://s3.amazonaws.com/S3BucketName-ignite-config/test-cluster.xml"
apacheignite/ignite I get a 403 which makes since as the S3Bucket is not
public:
o class org.apache.ignite.IgniteException: Failed to instantiate Spring XML
application context
[springUrl=https://s3.amazonaws.com/S3BucketName-ignite-config/test-cluster.xml,
err=IOException parsing XML document from URL
[https://s3.amazonaws.com/S3BucketName-ignite-config/test-cluster.xml];
nested exception is java.io.IOException: Server returned HTTP response code:
403 for URL:
https://s3.amazonaws.com/S3BucketName-ignite-config/test-cluster.xml]
If I try running it like this: sudo docker run -it --net=host -e
"CONFIG_URI=$CONFIG_URI" apacheignite/ignite it seems to start but does not
register it’s ip with the S3 Discovery bucket.
I'm a newbie and would appreciate this groups assistance! Please help me see
what I'm missing.
Thanks!
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ingite deploy in AWS
Posted by newigniter <to...@gmail.com>.
Its a while from this thread but I am having the same issue.
I host configuration file on S3 bucket. First I added this bucket to be
publically accessible and everything works correctly. I simply pass
https://s3.amazonaws.com/... link to my docker run command when I start
ignite(I am using docker) and it works ok. But do not want for my s3 bucket
to be public. I change this setting, and I add my ec2 instance the role
which whom I give s3 full access to it. I also create vpc endpoint allowing
access to s3 bucket. When I use aws-cli I am able to download s3 file to my
ec2 instance. But I cannot start container providing
http://s3.amazonaws.com/.. as configuration input source: It says access
denied. Basically, I cannot access to my s3 bucket without using aws-cli. I
tried using curl and I also get access denied.
Is there any workaround for this?
Tnx
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Ingite deploy in AWS
Posted by David Harvey <dh...@jobcase.com>.
As you already know, you cannot have the config file in the discovery
bucket, because discovery enumerates the contents of the discovery bucket.
But the credentials in the config bucket are not yet visible, so they
cannot be used. You can make the config bucket public, but not with the
key stored in it. But doing this briefly may help make sure that you are
on the right track.
We created a VPC endpoint, and gave that endpoint access to the config
bucket. The bucket policy we used to do this:
{
"Version": "2012-10-17",
"Id": "Policy1442576558687",
"Statement": [
{
"Sid": "Stmt1442576554133",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": [
"arn:aws:s3:::xxxx-config",
"arn:aws:s3:::xxxx-config/*"
],
"Condition": {
"StringEquals": {
"aws:sourceVpc": "vpc-b77954d2"
}
}
},
{
"Sid": "Access from JC Non-prod",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::11111111111:root"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::xxxx-config"
}
]
}
Test using the instance credentials
aws s3 ls s3://...-ignite-Config/
Uses no AWS credentials except that of the VPC endpoint
curl -s http://s3.amazonaws.com/...Config
We also use this in the config file
<bean id=“aws.creds”
class=“com.amazonaws.auth.DefaultAWSCredentialsProviderChain”/>
which requires the the instance role be able to access the discovery
bucket. But it gets rid of the exposed keys.
On Mon, Mar 26, 2018 at 12:28 PM, theizer <ti...@nrccua.org> wrote:
> I am trying to deploy an Apache Ignite cluster in our AWS VPC. I have done
> the following:
>
> • Created an S3 Bucket to store the ignite-config.xml file
> o Uploaded the config file to the bucket.
> Config file contents (just based on example in github):
> <beans xmlns="http://www.springframework.org/schema/beans"
> xmlns:util="http://www.springframework.org/schema/util"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="
> http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd
> http://www.springframework.org/schema/util
> http://www.springframework.org/schema/util/spring-util.xsd">
>
> <bean id="grid.cfg"
> class="org.apache.ignite.configuration.IgniteConfiguration">
> <property name="cacheConfiguration">
> <list>
> <bean class="org.apache.ignite.configuration.CacheConfiguration">
> <property name="name" value="IgniteCache"/>
> <property name="cacheMode" value="PARTITIONED"/>
> <property name="atomicWriteOrderMode" value="PRIMARY"/>
> <property name="writeSynchronizationMode" value="PRIMARY_SYNC"/>
> <property name="evictionPolicy">
> <bean
> class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
> <property name="maxSize" value="100000000"/>
> </bean>
> </property>
> <property name="swapEnabled" value="false"/>
> <property name="atomicityMode" value="ATOMIC" />
> <property name="backups" value="1" />
> </bean>
> </list>
> </property>
> <property name="discoverySpi">
> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
> <property name="ipFinder">
> <bean
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.
> TcpDiscoveryS3IpFinder">
> <property name="awsCredentials" ref="aws.creds"/>
> <property name="bucketName"
> value="S3BucketNAME-ignite-discovery"/>
> </bean>
> </property>
> </bean>
> </property>
> <property name="communicationSpi">
> <bean
> class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
> <property name="slowClientQueueLimit" value="1000"/>
> </bean>
> </property>
> </bean>
>
> <bean id="aws.creds" class="com.amazonaws.auth.BasicAWSCredentials">
> <constructor-arg value="ABCDEFGHIJKLMNOPRST" />
> <constructor-arg value="A1B2C3D4E5F6G7H8I9J1A1B2C3D4E5F6G7H8" />
> </bean>
> </beans>
> • Created an S3 bucket for S3 Discovery
> • Created an IAM role for Ignite-S3-Access with the
> AmazoneS3FullAccess
> policy attached and ec2.amazonaws.com as a trusted entity
> • Launched an EC2 instance from the Apache Ignite AMI for the
> us-east-1
> region
> o In my VPC private subnet
> o With IAM role created above for S3 Access
> o User Data:
>
> CONFIG_URI=https://s3.amazonaws.com/S3BucketName-
> ignite-config/ignite-config.xml
>
>
> When I log into the instance, the docker container is NOT running.
>
> I can verify that the IAM role for S3 is working by typing aws s3 ls and
> getting a list of all s3 buckets.
>
> If I try to manually start the docker container with the command: sudo
> docker run -it --net=host -e
> "CONFIG_URI=https://s3.amazonaws.com/S3BucketName-
> ignite-config/test-cluster.xml"
> apacheignite/ignite I get a 403 which makes since as the S3Bucket is not
> public:
> o class org.apache.ignite.IgniteException: Failed to instantiate
> Spring XML
> application context
> [springUrl=https://s3.amazonaws.com/S3BucketName-
> ignite-config/test-cluster.xml,
> err=IOException parsing XML document from URL
> [https://s3.amazonaws.com/S3BucketName-ignite-config/test-cluster.xml];
> nested exception is java.io.IOException: Server returned HTTP response
> code:
> 403 for URL:
> https://s3.amazonaws.com/S3BucketName-ignite-config/test-cluster.xml]
>
>
> If I try running it like this: sudo docker run -it --net=host -e
> "CONFIG_URI=$CONFIG_URI" apacheignite/ignite it seems to start but does
> not
> register it’s ip with the S3 Discovery bucket.
>
>
> I'm a newbie and would appreciate this groups assistance! Please help me
> see
> what I'm missing.
>
> Thanks!
>
>
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Disclaimer
The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.
This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.