AWS S3 FTP Access using s3fs
As you might know, AWS S3 is an object-based storage, not as native as block-based storage like EBS or EFS. But S3 has its power in lifecycle management, versioning, different storage classes and many more. But at times your existing clients who are comfortable with FTP/SFTP access may not have a clean way to upload files to S3.
To cover this, we have a few libraries that interface S3 and FTP. I am going to discuss a docker image that I built using s3fs(https://github.com/s3fs-fuse/s3fs-fuse)
My docker image provides more advanced features where it lets you map multiple users to multiple buckets or folders and has better permission system.
Let's assume you want to grant access to your S3 folder through your existing EC2 instance via SFTP.
Create an IAM Role for EC2 and grant access to S3 (either for a single bucket or full access)
Attach that role to the EC2 instance
Install docker on your EC2 instance (https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html)
docker pull selvamkf/s3-backed-ftp:new
Create env.list file
IAM_ROLE = name of role account linked to EC2 instance the container is running in.
AWS_ACCESS_KEY_ID = IAM user account access key.
AWS_SECRET_ACCESS_KEY = IAM user account secret access key.
USERS = Combination username, password, bucket mapping separated by double %. Multiple users can be created by using space in between.
Sample content for env.list,
IAM_ROLE=EC2-S3-Access USERS=myusername%%mypass%%mybucket myuser2%%pass2%%bucket2
When you use IAM_ROLE, the other keys are not required. i.e AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. If you want to map to individual folder inside a bucket, you should give as,
docker run -it -p 24:22 --privileged --env-file /home/ec2-user/env.list selvamkf/s3-backed-ftp
This command starts a new container and takes env.list file as input. Make sure to run it on a screen. Each user you have given in the input should have SFTP access now. Note, we have mapped port 24 of the host EC2 instance to port 22 of docker. So you need to use port 24 for your SFTP connection. You need to ensure the EC2 instance security group allow port 24 for your people to use SFTP.
Additional information for AWS ECS
The same setup can be run from AWS ECS, you need to give IAM_ROLE, USERS as environment variables of the container.