This site is just simple generated HTML files that I host on AWS. It is delivered by Cloudfront, which is a network of servers around the word so that requests can be handled by a server nearby rather than crossing oceans. This makes things speedy and also dirt cheap as html websites are teeny tiny, so not much to store or transfer.
Below are the basic instructions so you can set this up for yourself. These instructions are also for me, so next time I want to setup a new site and have completely forgotten how to do this, I have something to refer to.
We'll be creating two S3 buckets, one to store the website files and one to store the logs.
For the website bucket, the first thing is to give it a useful name, I'll be referring to it as [WEB_BUCKET_NAME] through the rest of this post. You'll also need to uncheck the box that blocks all public access, this is fine since the website is going to be public anyway. You should use the web interface here to upload a file to the bucked as well, preferably your default page, index.html. This file we can use to test that things are working before we are setup to upload the whole site.
Then you'll need to go in and tweak some properties, in particular at the bottom there is the option for static website hosting. Enable that and set the index document to match your default page,index.html. Even tho we will be using Cloudfront, that can serve everything directly from the bucket without this enabled, we want to use it as it handles default documents better.
The box we checked when creating the bucket only stopped blocking public access, we still need to enable it. To do that we need to go to the permissions tab and set this bucket policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadForGetBucketObjects",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::[WEB_BUCKET_NAME]/*"
}
]
}Now in theory your bucket should be available over http, you can confirm that by pointing a browser to:
http://[WEB_BUCKET_NAME].s3-website-us-east-1.amazonaws.com/
This url should also be available at the bottom of the properties tab, now that we have enabled static hosting.
Now we can setup the S3 bucket to store logs. It's much easier than the website bucket, just give it a name [LOG_BUCKET_NAME], and leave public access blocked.
We also need a user account to be able to upload files and download logs from our two buckets, so head over to IAM and start creating a new user. Pick a name and select the check box for "Access key - Programmatic access". We won't be setting any any permissions at this point, we will edit the user later to add them directly, so just keep clicking next and eventually your user will be created.
You'll receive a Access key ID and Secret access key, make sure you record these for later, we'll refer to them as [S3_KEY] and [S3_SECRET].
Now jump back to the users list and click on your newly created user, then click "Add inline policy", click the JSON tab then you can copy and paste in these permissions:
{
"Version": "2012-10-17",
"Statement": [
{
"Resource": [
"arn:aws:s3:::[WEB_BUCKET_NAME]",
"arn:aws:s3:::[WEB_BUCKET_NAME]/*",
"arn:aws:s3:::[LOG_BUCKET_NAME]",
"arn:aws:s3:::[LOG_BUCKET_NAME]/*"
],
"Sid": "Stmt1464826210000",
"Effect": "Allow",
"Action": [
"s3:DeleteObject",
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket",
"s3:PutObject"
]
}
]
}Click review policy, give it a name then finally click create policy. Now our user should be all setup to upload and download from our S3 buckets.
With an account setup we can create a simple bash script to upload our HTML and assets to the server.
We'll be using the AWS CLI tools. On Debian or Ubuntu this can be installed with apt:
sudo apt install awscli
Then we can create a simple script to sync our local copy of the website to the S3 bucket:
#!/bin/bash
export AWS_ACCESS_KEY_ID=[S3_KEY]
export AWS_SECRET_ACCESS_KEY=[S3_SECRET]
export AWS_DEFAULT_REGION=us-west-2
aws s3 sync [LOCAL_WEB_DIR] s3://[WEB_BUCKET_NAME]/ --deleteWhere [LOCAL_WEB_DIR] is the folder on your local disk, something like ./html/.
The --delete switch will delete anything in the bucket that is not in your local dir. This means that after the upload completes, the S3 bucket will be a perfect mirror of your local dir.
And with that your website should be up and running at the URL we tried earlier.
Now it's time to get things running from our own domain, with https and over the CDN. So head over to the Cloudfront section of the AWS dashboard and create a new distribution.
Firstly select your origin, this is the S3 bucket you have already setup, and it should be listed in the drop down box. You can leave most settings as the default all the way down untill "Alternate domain name", here you want to add the domain name you want to use for your site and select a SSL certificate for it. If you don't have one already, click request certificate and follow that process.
Lower down you can enable standard logging, and can select the bucket you created earlier to store the logs.
After completing creation you should be able to access this distribution by it's internal domain name, something like abc123.cloudfront.net.
Now inside the DNS settings of your domain, you can set the CNAME to be the cloudfront internal domain name from above. Once the dns updates (this can take a while depending on your previously set TTL settings) going to your domain in a web browser should bring up your website!
Similarly to the upload script, we can make a log download script to download our logs from the server.
Also we can use GoAccess to process the downloaded logs and generate a HTML report.
On Debian or Ubuntu it can be installed via apt:
sudo apt install goaccessFor information about installing on other platforms, check here.
From here we just need a script to download the logs and generate the report:
#!/bin/bash
export AWS_ACCESS_KEY_ID=[S3_KEY]
export AWS_SECRET_ACCESS_KEY=[S3_SECRET]
export AWS_DEFAULT_REGION=us-west-2
aws s3 sync s3://[LOG_BUCKET_NAME]/ ./log/
zcat log/*.gz | goaccess --log-format CLOUDFRONT --date-format CLOUDFRONT --time-format CLOUDFRONT -o report.htmlAfter running that script you can check out report.html to see all the pretty graphs and stats from your log.
Previous: Optimizing Online Game Leaderboards