A successful eCommerce website must face the inevitable scaling task as traffic increases. However, scaling servers and web application is difficult. IT teams know this, but often management doesn’t understand how difficult managing site uptime, scaling, and productivity of the site can really be. Scaling is essential to the development of businesses, so let’s take a closer look at the steps involved.
What is scalability and server scaling?
Every website, even using cloud or hybrid storage, is linked to a physical server or group of servers, known as a server farm. The servers provide computing power for websites to display data to traffic.
Scalability is the ability of a system to manage increased traffic as a website grows flexibly. To serve higher traffic flows, web applications need increased computing power from the servers, and the applications’ components also need to scale.
Server scaling means adjusting the server computing power, usually to increase capacity. This ‘scaling up’ can be done either vertically, replacing the server with a larger, more powerful one, or horizontally, which means adding several servers and combining their total computing power.
What is autoscaling?
Autoscaling means automatically adjusting computing power according to load. It is a system whereby the power is increased as the load is increased. Server capacity should automatically increase as traffic levels rise. It also works in reverse; as traffic decreases, so does server capacity, saving on computing costs during low-traffic times.
Autoscaling allows systems to be flexible, reducing the need to monitor and predict traffic levels. This automatic adjustment aims to save power for when it’s needed and reduce costs for business owners.
However, autoscaling doesn’t actually work in such simple terms.
Why is scaling so difficult?
Servers need to provide a stable and consistent service. Businesses scale their websites to function optimally, even during periods of heavy traffic.
But scaling servers doesn’t mean the website will scale equally as well. The architecture of a website will determine how difficult it is to maintain performance under increased demand.
Web applications generally encounter four main problems when scaling:
- Search
- Concurrency
- Consistency
- Speed
To illustrate this, an analogy of a phone book can be used. Imagine an office of 20 people, some of whom have phone numbers. There might be 15-20 names listed beside a corresponding phone number.
Finding the name and phone number of whoever you need to reach is easy, as the data is listed on a single sheet and distributed to all workers in the office.
Search Problem
The search problem refers to the difficulty of finding information in large data sets. The larger the data set, the more intensive the search to find the required information.
If the office phone book were to expand to include 1 million workers, it would take hours to find a specific phone number if names and numbers are not organized in a searchable way. Whether it is alphabetical, geographical, or numerical, there needs to be a strategy to deal with the search problem.
Concurrency Problem
The concurrency problem is the difficulty of making data available to several people, programs, or resources simultaneously.
So if all 1 million people in our office analogy wanted to use the same singular phone book at the same time, there would be a massive wait time, and most people would never get access to the data. Additional copies of the phone book (making copies is called “replication”) would help ease the concurrency problem as long as distribution, ensuring everyone has a phone book, would ease the problem. However, there are two remaining issues that need to be solved.
Consistency Problem
The difficulty of constantly updating data is the consistency problem. If a phone book is issued with updates, by the time distribution takes place, some of the data will already be out of date, with people changing phone numbers or even leaving the company. The data is not always accurate and available, and this leads to issues with consistency.
Speed Problem 
As an increasing number of requests or transactions are logged, the speed of answering those requests slows. Adding changes increases the time needed to search data. If in the office example, the company were to change phone companies, this would mean numbers would change, and a huge amount of data would need to be updated in as fast amount of time as possible.
Scaling an eCommerce Site
These four issues are problems when scaling eCommerce websites. ECommerce sites that expand from a few products to thousands of products require far more advanced search functions and meaningful categories to help customers find what they need. The more products, the more your application requires from the server.
Most open-source eCommerce platforms are built around inventory truthfulness, prioritizing consistency over concurrency because both do not operate at the same time.
Planned online sales, like Black Friday in North America, drive website traffic to higher-than-average volumes. This causes speed problems and often crashes eCommerce sites, particularly for big box stores.
Systems engineers use hundreds of interlocking algorithms to try to maintain a site’s uptime. However, when the site needs to scale and respond to demand very quickly, this is when the algorithms fail, and a site crashes, sending traffic to 404-page errors or slowing to the point of not responding.
The Cost of Vertical vs. Horizontal Scaling 
Scaling an entire web application is incredibly complex. The costs can quickly escalate as project issues arise. It is important that business owners understand how costs can rise so quickly.
Vertical Scaling Costs
When vertical scaling requires increasingly higher-end servers, costs increase exponentially. The difference in power is reflected in the power of the server. So a simple setup with limited capacity might cost $100 per year, while a high-performance server will cost $10,000 per year.
Businesses that need to replace a physical server for a higher-end one also need to factor in the costs of downtime to make the switch. Vertical scaling doesn’t work cohesively with autoscaling because scaling always requires downtime.
Horizontal Scaling Costs
Horizontal scaling, or scaling out, is the cheaper scaling option, as hardware costs are greatly reduced. Datacenter costs can increase with the need for more space, cooling, and power usage. Added to the costs are licensing fees for nodes, which can escalate. Businesses can be charged annual licenses for servers even if they only use them for a few days of the year.
Another factor to consider is the hidden costs of having dedicated expertise needed to set up and maintain the autoscaling and application scaling. Dedicated IT teams consist of devoted DevOps engineers who work to autoscale sites even under heavy load.
Stop-gap measures can improve performance, but solutions are short-lived, with bugs appearing as code becomes more complex. Many bugs aren’t even problematic until there’s a high load on your system. For example, an IT team might shard data to improve performance. That could be as simple as all products with SKU code starting A-J is in server 1 and K-Z is in server 2. If a product start C is in high demand, traffic will flow through server 1, causing it to fail earlier than expected.
So while horizontal scaling provides greater flexibility for autoscaling and has lower administrative costs than vertical scaling, hidden resource costs can quickly add up.
Difficulty Autoscaling eCommerce Sites
It is not possible to simply scale all off-the-shelf software products. Websites are made up of cached and dynamic content.
- Cached content includes images, text, files, etc., such as Home or Contact Us pages. These pages can be pre-loaded using a content delivery network (CDN) so servers don’t have to deliver the content to each new visitor.
- Dynamic content is unique to the site visitor and depends on user actions. For example, adding a product to a checkout cart creates dynamic content. The back-end functionality checks inventory updates the database and changes the cart icon to show the number of products it contains.
Dynamic content is critical to the customer journey. Each change to dynamic content informs the customer; however, on all eCommerce sites, the inventory system and the payment gateways tend to slow down page responses.
Inventory systems preserve order and stock volume databases. When under high traffic, shopping carts and inventory systems become stressed, harming website performance.
Payment gateways can cause larger problems. Third parties always control payment gateways. This means that payment processors work with other websites too. This can mean that at times of high volume sales, such as the Christmas sales period, payment processors can be in high demand, resulting in slower than normal processing. Payment gateways are the point of failure for numerous eCommerce sites.
Website Scaling Can Fail
When customers are all online at once, traffic overwhelms the site. Despite autoscaling, bottlenecks occur on the user journey producing errors and crashing sites. Some of the biggest online stores in the North American market have failed to meet traffic demands on some of the most important online sales days. Many sites failed on the US’ cyber Friday sales in 2021 despite autoscaling installations and IT technicians prepared to deal with high-volume traffic. Even in 2023, retailers are struggling to maintain performance on eCommerce sites.
Scaling your website is a necessary step in meeting demand and maintaining a viable business. However, as discussed, there are options, and each provides benefits and drawbacks. Discussing your options and having a professional assess your site and its capabilities is essential to making decisions that grow your business.
