qwd

Moving from the attic to the cloud

This web page (and most of the other projects that I have had) have up until today been run on an old, slow, and dusty family desktop computer placed in the attic at my home. There are several advantages of this approach: It is cheap (relative to the performance I get from it), good bandwidth, and large storage. Having physically access to the server have also been an advantage for me for learning server management. There are several times I have had to sit almong old shoes and storage boxes up in the attic to debug how I managed to lock my self out of the computer via ssh or why an update completly broke the network connection after a reboot. I am happy for all the experience I have gained over the last few years, but it is time to move on.

The main issue I have had is that it is not always easy to ensure reliable availability. Although it is not a requirement from anywhere or anyone, I think it is a nice to have. I legitimately want the users of my projects to have a nice experience. That meens no ads, no slow loading times and bloat, and no downtime. With this in mind I have decided to move all my infrastructure over to the cloud. I have choosen AWS as my preferred provider.

The good and the bad
Cloud done right is good. Cloud done wrong is really bad. AWS usually brand their product in a way that is supposed to make me think that I "only have to think about the application code, and not provisioning and managing servers". It is framed in a way that makes it easy to conclude that you dont have to think about the infrastructure. This could not be further from the truth. I have never have had to be more carefull whith the infrastructure than working with AWS (or any cloud provided, really). Sometimes you pay for what you provision (e.g. provisioned R/W units in DynamoDB), other times you pay for what you, or the users of your systems, use (e.g. egress traffic from an EC2 instance). And when you pay for what you use with no (realistic) limit on how much you can use, a bad config or a recursice setup somewhere can be really expensive. There are several horror stories out there when the pay-for-what-you-use pricing model has gone terribly wrong for the poor cloud manager.

Luckily for me; I know what I am doing™
Since I don't have any business plan related to any of my projects, focusing on cost management is my top one priority. This, together with hight security, high availability, low latency and high scalability. There are dosens of different approaches one can take to serve and host a web application on AWS. AWS recommends applying cloud native solutions and principles for new applications. This is cool, as the cost/performance/scalability aspects of the solution is really attractive. However, it does come with a quite heavy vendor lock-in and to some degree add a few black-box solutions here and there which are difficult to debug.

As most of my projects are not cloud native in any way there will be a transition period where a cheap EC2 t4g.nano instance will be used the same way as the dusty asus attic server while I slowly offload certain workloads to more cloud native solutions. As an example: My 2000meter.no project (that is a classic always 75% finished project) will have three endpoints: www.2000meter.no, cdn.2000meter.no, and api.2000meter.no. The www-subdomain will point to the ec2-instance behind CloudFront (for now), the cdn-subdomain will point to an s3 bucket through CloudFront (possible permanent), and the api-subdomain will go though API Gateway for authorization and from there passed on to a Lambda function with a DynamoDB storage solution.

I thave high hopes for this, and I think it can be fun to explore the possibilities that (carefully designed) cloud native solutions will provide in terms of reliability, scalability and cost optimization.

About Movieshrimp

Do you have problems picking a good movie for the night? You don't know what to watch and are tired of endless scrolling on Netflix? We know the struggle. MovieShrimp gives you three good movie recommendations. Honestly, you can’t really go wrong with any of these. All three are high quality movies that are publicly appreciated. Just pick one and jump right in! We give you three good recommendations every day.

Take a quick look at the genres and severity score to get an indication of the level of disturbing scenes such as violence, drugs, strong language, nudity, intense and frightening scenes. The value of the severity score range from 1 (very low) to around 5 (very high) and is based on the public's opinion on the different disturbing topics mentioned above. The score must be seen in relation to what is expected given the genres to the movie. Keep in mind that this is just as a guidance and that everyone react to disturbing scenes in different ways.

Technical challenges
In addition to actually find good movies to recommend, the site had several other challenges that had to be solved. I wanted the site to be virtually free to run, fast and accurate independent on where in the world the user is located and how many that tries to access the page at the same time. The site should be free to use and free of ads. To solve all these requirements, the site is placed behind Cloudflare and served from an AWS S3 bucket. Since the movie recommendations is rotated once a day, that will allow for some heavy caching og even html and have a periodic AWS Lambda function that run once a day and choose the new recommendations in a semi-random way. All this ends up costing less than NOK 4 per month independent of user count.

The name; qwd

The three letter combination qwd was made by simply tyipng three random letters on a standard qwerty keyboard. Initially, the letters in itself did not mean anything in particular and was only chosen for their convenience with regard to typing them out. However, I felt that they would have more meaning if the letters was an abbreviation of something. I know it is a bit unusual to start with the abbreviation and find a fitting name than the other way around, but that is what I would try to do. There are several candidates, but none of them are good. A few of them are listed below:

• quickest way down
• quiet water design
• quest without destination
• quark with dignity
• quintillion water droplets

The names are not excellent. If you have any better ideas and want to share it with me, please send me an email and I will be glad to add it to the list of potential names. For the moment, qwd will just be qwd. Nothing less, nothing more.

New domain name

qwd.no is the new domain name and has replaced the old domain spaghettiprojecti.no. I felt that I needed a name that was easier to remember and easier to type, hence the change. The old domain will redirect to the new one until it is unregistered which will be around March 2019.

Note about the current state of NTNU exercises and solutions

Due to a change in how NTNU distributes exercises and solution affecting from around January 2018, the auto-fetch program that previously fetched and organised everything on this site is no longer working. As the situation is now, I am sorry to say that this cannot be fixed. I have suspended the developent of the project for the moment. I will leave the archive as it is in its current state, as I believe it could be of value for others and I think it is one of the largerst one of its kind. I might add more manually later, but I cannot say what or when.

It was a good run.

Ending note

Out of curiosity: how did you get here? No one visits this site exepts me and various bots :/