blog-banner

AWS S3 FTP Access Using s3fs

  • AWS ECS
  • Aws S3
  • AWS S3 FTP
  • FTP
  • S3FS

S3 FTP

As you might know, AWS S3 is object-based storage, not as native as block-based storage like EBS or EFS. But S3 has its power in lifecycle management, versioning, different storage classes, and many more. But at times your existing clients who are comfortable with FTP/SFTP access may not have a clean way to upload files to S3.

To cover this, we have a few libraries that interface S3 and FTP. I am going to discuss a docker image that I built using s3fs(https://github.com/s3fs-fuse/s3fs-fuse)

s3-backed-ftp which is based on https://hub.docker.com/r/factual/s3-backed-ftp/

My docker image provides more advanced features where it lets you map multiple users to multiple buckets or folders and has a better permission system.

Let's assume you want to grant access to your S3 folder through your existing EC2 instance via SFTP.

Step 1: 

Create an IAM Role for EC2 and grant access to S3 (either for a single bucket or full access)

Step 2

 Attach that role to the EC2 instance

Step 3

Install docker on your EC2 instance (https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html)

Step 4

docker pull selvamkf/s3-backed-ftp:new

Step 5

Create env.list file

  1. IAM_ROLE = name of role account linked to EC2 instance the container is running in.

  1. AWS_ACCESS_KEY_ID = IAM user account access key.

  2. AWS_SECRET_ACCESS_KEY = IAM user account secret access key.

  3. USERS = Combination username, password, bucket mapping separated by double %. Multiple users can be created by using space in between.

Sample content for env.list,

IAM_ROLE=EC2-S3-Access
USERS=myusername%%mypass%%mybucket myuser2%%pass2%%bucket2


When you use IAM_ROLE, the other keys are not required. i.e AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. If you want to map to an individual folder inside a bucket, you should give as,

myusername%%mypass%%mybucket:/parent/child
 

Step 6

docker run -it -p 24:22 --privileged --env-file /home/ec2-user/env.list selvamkf/s3-backed-ftp


This command starts a new container and takes env.list file as input. Make sure to run it on a screen. Each user you have given in the input should have SFTP access now. Note, we have mapped port 24 of the host EC2 instance to port 22 of docker. So you need to use port 24 for your SFTP connection. You need to ensure the EC2 instance security group allows port 24 for your people to use SFTP.

Additional information for AWS ECS

The same setup can be run from AWS ECS, you need to give IAM_ROLE, and USERS as environment variables of the container.