It's not that hard making your web site scalable
I visited Milk & More today to see what they sell, having heard their adverts serveral times on the radio.
I was met with this "queue". Yes, that's right, "When it is your turn, you will have 10 minutes to enter the website." What?
Apparently, queue-it have been in business for over 9 years and offer this gateway onto your web site in case it is struggling. I am dubious about the un-substantiated claims about how damaging it is if people cannot access your site because it is busy. However, I don't personally find the solution very enticing or friendly. At the bottom of the page, it says my wait is over an hour because of 27K people who want to visit the site.
This is smelly really. Imagine going shopping and a shop is really busy. Is it really much better to be told to come back in an hour when you will be allowed into the shop as opposed to trying to struggle in? Not really.
The real problem here is that 27K people/hour is really not very many for even a basic web site that has some simple optimisations built-in. The problem is that people are not developing sites to be even slightly scalable, even with simple of-the-shelf tools/techniques and they end up resorting to these slightly nasty workarounds. At SmartSurvey, we have had over 8000 concurrent users filling in surveys with little degredation and we certainly don't do that much that is very clever other than measuring and improving. This was running across 4 web servers. We now have 6.
How do we make web sites scalable?
This is usually really easy. In most cases you simply need to think about sharing session (if you even need it) and in the case of dotnet, and possibly others, you need to ensure that the keys to sign XSRF tokens are shared across the web server farm. You also need some kind of load-balancer in front, ideally a hardware dedicated machine.
Now you should almost never be hosting your site internally. Why do that? To do it properly, you need an air-conditioned server room with UPSs and redundant hardware. In most cases, unless you are a big dollar enterprise, you are better off using the cloud or a rackspace-type provider. You need to make sure that you can easily add additional web servers to your V-net with minimal effort. In the cloud, this can be anything from seconds to minutes but even with 'rackspace', you should be able to provision another virtual server in less than 30 minutes.
Provisioning machines quickly on the hosting platform is of limited use if you then have to spend a day installing and configuring things.
Most platforms have pre-configured VM appliances (e.g. Windows + IIS) and others have more complex solutions like being able to build an entire image and save it for duplication later. If you use Docker or another container tool, this is even easier, although you need to learn the basics of Docker first.
You can also mix CMS with static nowadays so e.g. you add/edit your pages and then press a button to generate static files from the CMS and serve these from the front-end server. Even if you have some dyanmic content, you can still use static content in a few places like the front page where you can try and quickly filter our any "web tourists" who just want to know what you do in simple terms and won't continue to eat your server resources.
For example, if your catalog doesn't change much, why not cache frequent search terms to avoid a large hit on the database. Instead, return the cached content. Even reducing your heaviest queries by 50% could potentially make your site run super-quickly when it would otherwise fall over. You can use level 2 caching of database results using a library relevant to your data layer or otherwise use something really simple like redis, memcache or a built-in memory cache in your web application framework. Just remember, memory is volatile so you mustn't rely on the data being available from cache, it needs to be reloadable if it gets purged/lost.
There are many free or paid-for tools that can monitor the performance of your app and let you know which bits are slow. Work on those and you might not need to touch anything else. Some examples are Application Insights, New Relic and Dynatrace. Some of these are really valuable (for a price!) and can sometimes identify individual SQL queries that are slow. They can also alert you due to system overloading due to a spike in traffic or a server problem.
There are loads of improvements you can make on a database but the basic understanding of using indexes well is the biggest difference between OK and disaster. An index can reduce the time a query takes literally by hundreds of times with no loss of information! Imagine what difference that might make.
These sites are terrible even when they are not overloaded. How much worse will they be with 1000 people on the site? If you do not know how to measure and optimise web sites then learn, it is not hard. Also, you need to know what you are embedding on your site and you need a strong engineering team who can push back on the myriad scripts that everyone wants to embed. You also need to review these regularly and dump anything you don't need - you might already be breaking the law if these are not documented somewhere!
I was met with this "queue". Yes, that's right, "When it is your turn, you will have 10 minutes to enter the website." What?
Apparently, queue-it have been in business for over 9 years and offer this gateway onto your web site in case it is struggling. I am dubious about the un-substantiated claims about how damaging it is if people cannot access your site because it is busy. However, I don't personally find the solution very enticing or friendly. At the bottom of the page, it says my wait is over an hour because of 27K people who want to visit the site.
This is smelly really. Imagine going shopping and a shop is really busy. Is it really much better to be told to come back in an hour when you will be allowed into the shop as opposed to trying to struggle in? Not really.
The real problem here is that 27K people/hour is really not very many for even a basic web site that has some simple optimisations built-in. The problem is that people are not developing sites to be even slightly scalable, even with simple of-the-shelf tools/techniques and they end up resorting to these slightly nasty workarounds. At SmartSurvey, we have had over 8000 concurrent users filling in surveys with little degredation and we certainly don't do that much that is very clever other than measuring and improving. This was running across 4 web servers. We now have 6.
How do we make web sites scalable?
Web Server Farm
Firstly, your site needs to be multi web-server friendly even when you only expect to use 1 server. You should also use some kind of hosting that allows a relatively easy scale out of web servers.This is usually really easy. In most cases you simply need to think about sharing session (if you even need it) and in the case of dotnet, and possibly others, you need to ensure that the keys to sign XSRF tokens are shared across the web server farm. You also need some kind of load-balancer in front, ideally a hardware dedicated machine.
Now you should almost never be hosting your site internally. Why do that? To do it properly, you need an air-conditioned server room with UPSs and redundant hardware. In most cases, unless you are a big dollar enterprise, you are better off using the cloud or a rackspace-type provider. You need to make sure that you can easily add additional web servers to your V-net with minimal effort. In the cloud, this can be anything from seconds to minutes but even with 'rackspace', you should be able to provision another virtual server in less than 30 minutes.
Automating Deployments
To make the most of the web server farm, your deployments should be automated. This is really easy with closed and open sourced solutions like Azure Devops, Jenkins, Octopus and many others.Provisioning machines quickly on the hosting platform is of limited use if you then have to spend a day installing and configuring things.
Most platforms have pre-configured VM appliances (e.g. Windows + IIS) and others have more complex solutions like being able to build an entire image and save it for duplication later. If you use Docker or another container tool, this is even easier, although you need to learn the basics of Docker first.
Static Content
A lot of content on a web site can be static. Although we love the idea of the CMS to quickly and easily change things without involving IT, a static page is much lighter on the server and can be relatively heavily cached on public caches as well as the server itself.You can also mix CMS with static nowadays so e.g. you add/edit your pages and then press a button to generate static files from the CMS and serve these from the front-end server. Even if you have some dyanmic content, you can still use static content in a few places like the front page where you can try and quickly filter our any "web tourists" who just want to know what you do in simple terms and won't continue to eat your server resources.
Caching Dynamic Content
Caching is relatively cheap and easy and as long as you cache things that are genuinely reused heavily, you get a lot of pros for not many cons.For example, if your catalog doesn't change much, why not cache frequent search terms to avoid a large hit on the database. Instead, return the cached content. Even reducing your heaviest queries by 50% could potentially make your site run super-quickly when it would otherwise fall over. You can use level 2 caching of database results using a library relevant to your data layer or otherwise use something really simple like redis, memcache or a built-in memory cache in your web application framework. Just remember, memory is volatile so you mustn't rely on the data being available from cache, it needs to be reloadable if it gets purged/lost.
Measure Performance
Don't guess what is slow on your site. It isn't always what you think and it might be a single operation is killing everything either because it is very slow or maybe it is simply called so many times across the site it affects the overall performance.There are many free or paid-for tools that can monitor the performance of your app and let you know which bits are slow. Work on those and you might not need to touch anything else. Some examples are Application Insights, New Relic and Dynatrace. Some of these are really valuable (for a price!) and can sometimes identify individual SQL queries that are slow. They can also alert you due to system overloading due to a spike in traffic or a server problem.
Understand your database
Some developers have little or no database training but are expected to know how to write web applications that are frequently accessing one and possibly hammering it at the time.There are loads of improvements you can make on a database but the basic understanding of using indexes well is the biggest difference between OK and disaster. An index can reduce the time a query takes literally by hundreds of times with no loss of information! Imagine what difference that might make.
Stay off the Marketing Junk
Some web sites are absolutely terrible for marketing and tracking scripts. I can imagine 2 or 3 different things like Hubspot and Google Tag manager but some sites literally have 100s of requests made off of a single page.These sites are terrible even when they are not overloaded. How much worse will they be with 1000 people on the site? If you do not know how to measure and optimise web sites then learn, it is not hard. Also, you need to know what you are embedding on your site and you need a strong engineering team who can push back on the myriad scripts that everyone wants to embed. You also need to review these regularly and dump anything you don't need - you might already be breaking the law if these are not documented somewhere!