Art of scalability

I will write two or three articles about scalability. I will call the first one ‘Scalability principles’ and I will speak about what is the scalability and its principles, and then I will move to Scalability pattern, anti-pattern, and guidelines in next posts. So let’s start.

Go to : Part 1- Scalability principles
Scalability Guidelines for building scalable software system:

1. Decrease processing time:

In any optimization book, or course you will found that code optimization is about decrease processing time using better algorithms, code, design, or better strategy. Also decrease – request – processing is one of main Scalability guidelines. To increase the amount of work that an application does is to decrease the time taken for individual work units to complete. For example, decreasing the amount of time required to process a user request means that you are able to handle more user requests in the same amount of time. But how, here are some examples:

• Caching: if the data and the code can’t be collocated, cache the data to reduce the overhead of fetching it over and over again.
• Parallelization: decrease the time taken to complete a unit of work by decomposing the problem and parallelizing the individual steps.
• Remoting: reduce the amount of time spent accessing remote services by, for example, making the interfaces more coarse-grained. It’s also worth remembering that remote vs local is an explicit design decision not a switch and to consider the first law of distributed computing – do not distribute your objects.
• Collocation: reduce any overheads associated with fetching data required for a piece of work, by collocating the data and the code.
• Pooling: reduce the overhead associated with using expensive resources by pooling them.

We always try to introduce abstractions and layers that can make the previous techniques easy when are required. But between the highest abstractions and lowest abstraction usually the software developers fall. Yes, these concepts are great tools for decoupling software components, but they have a tendency to increase complexity and impact performance, particularly if you’re converting between data representations at each layer. Therefore, the other way in which processing time can be minimized is to ensure that the abstractions aren’t too abstract and that there’s not too much layering. In addition, it’s worth understanding the cost of runtime services that we take for granted because, unless they have a specific service level agreement, it’s possible that these could end up being the bottlenecks in our applications.

What I want to say Is: you don’t need to implement all the previous points, because one size don’t fit…

2- Partition:

Decreasing the processing time associated with other related concepts, such as Code Optimization as I said before, Partition, Asynchronous.

you partition your work by Functional decomposition/Partition by function ( When related pieces of functionality belong together, while unrelated pieces of functionality belong apart)  Further, the more decoupled that unrelated functionality can be, the more flexibility you will have to scale them independently of one another. For example single database server sitting behind many web servers. When that starts to become the bottleneck, you have to change your approach and one way to do this is to adopt a partitioning strategy. Put simply, this involves breaking up that single piece of the architecture into smaller more manageable chunks. Partitioning that single element into smaller chunks allows you to scale them out and this is exactly the technique that large sites such as Amazon, Facebook use to ensure that their architectures scale. Partitioning is a good solution.

3- Asynchronously:

After you partition your application, the next key element to scaling is the aggressive use of asynchrony. If component X calls component Z synchronously, X and Z are tightly coupled, and that coupled system has a single scalability characteristic — to scale X, you must also scale Z. Equally problematic is its effect on availability. Going back to Logic 101, if X implies Z, then not-Z implies not-X. In other words, if X is down then Z is down. By contrast, if Z and X integrate asynchronously, whether through a queue, multicast messaging, a batch process, or some other means, each can be scaled independently of the other. Moreover, Z and X now have independent availability characteristics – A can continue to move forward even if B is down or distressed.

This principle can be applied up and down an infrastructure. Techniques like SEDA (Staged Event-Driven Architecture) can be used for asynchrony inside an individual component while retaining an easy-to-understand programming model. Between components, the principle is the same — avoid synchronous coupling as much as possible. More often than not, the two components have no business talking directly to one another in any event. At every level, decomposing the processing into stages or a phase, and connecting them up asynchronously, is critical to scaling

4- Scalability is about concurrency:

As Researcher in Parallelism, I can say Scalability is about concurrency and the concurrency is about the parallelism . Using the parallelism to decrease the processing time, especially if your work have asynchronously flow, is great. But remember the parallelism is not that easy. But there are some simple advices that can help to decrease the processing time using parallelism:

• If you do need to hold locks (e.g. local objects, database objects, etc), try to hold them for as little time as possible.
• Try to minimize contention of shared resources and try to take any contention off of the critical processing path (e.g. by scheduling work asynchronously).
• Any design for concurrency needs to be done up-front, so that it’s well understood which resources can be shared safely and where potential scalability bottlenecks will be.