A simple explanation for Amdahls law
Amdahls law impacts our daily lifes without many people being aware of it. Although it is mainly associated with parallel computing it is much more universal. It is one of the fundamental laws of nature. This article explains it in simple terms.
Think about an anthill. There are two methods that an ant can use to build it. If that ant is somewhat selfish it would probably try to build it alone. However ultimately it would run out of time. Obviously it is more intelligent to work together with an entire workforce of ants. Doing that will speed up completion of the anthill dramatically. But that approach is limited as well. No matter how big that workforce is they will never be able to build it within the instant of a second. The size of the colony is exactly predetermined because they are aware of Amdahls law.
It is the same thing with every human endeavour. We have two resources that we can use: Time and space (workforce). Time is a limited resource by its very natre while space is virtually unlimited. But as everyone knows: Rome wasn’t built in a day and that is because of Amdahls law.
So what does Amdahls law (after Gene Amdahl) state?
The maximum possible speedup of a task that can be achieved through parallelization is determined by the serial portion of that task. An ant trail simply does not have multiple lanes. It is a trail not a highway.
We can calculate the speedup of a task by knowing what portion of it always has to be executed in serial. Assume we have a task that needs 10 hours to complete if a single worker does it. We are able to parallelize 90% of the job.
Buy using two workers we can do 90% of it (9 hours) in half the time. The 10% (1 hour) go on top of it. So we have 4.5 parallel execution + 1 hour serial execution.
Let’s calculate that for bigger workforces and see what we get.
1 worker: 10 hours 2 workers: 4.5 hours + 1 hour = 5.5 hours 3 workers: 3 hours + 1 hour = 4 hours 4 workers: 2.25 hours + 1 hour = 3.25 hours 5 workers: 1.8 hours + 1 hour = 2.8 hours 10 workers: 0.9 hours + 1 hour = 1.9 hours 100 workers: 0.09 hours + 1 hour = 1.09 hours 1000 workers: 0.009 hours + 1 hour = 1.09 hours 10000 workers: 0.0009 hours + 1 hour = 1.0009 hours
No matter how many workers we use. We will never be able to complete that task in less than an hour. Our only way to deal with it is to reduce the serial portion of it. We’ll go into more detail in further articles.
If that article was helpful to you, check out The IT Automation Blog on a regurlar basis. There is one new article every single day and in upcoming articles we will go more into detail on how to deal with Amdahls law.
