The Power of Simplicity: How a random iterator saved the day

 …and my database.

How many times has something so simple saved the day? Sometimes, the simplest solutions can have a significant impact on a problematic situation. In this blog post, we'll explore one such scenario where a small algorithm change led to substantial performance improvements in an existing system.

The problem 

The issue at hand involved a component in a system using a database table as a makeshift queue for processing updated rows. Multiple processor instances read from the same table, with Redis locking in place to prevent concurrent processing of the same row. However, the processors working in the same order led to numerous collisions and timeouts, causing the system to slow down. 

As illustrated in the image, the processors were interfering with one another; while they ultimately completed the work, excessive time was consumed in fruitless attempts to lock records, which consequently increased the strain on the database.

The solution: A Random Iterator

A random iterator is an algorithm that iterates over the elements of a collection in random order. Here is a C# extension method for such an iterator over IList<T>.

By implementing a random iterator in the component, each processor would grab a full copy of the table and process it randomly, significantly reducing the chances of locking collisions.  The table's size allowed it to fit entirely into memory, and idempotence ensured that duplicated attempts were properly handled. 


The result was a drastic performance improvement, allowing the number of processor instances to be reduced, and the overall solution went from a system barely performing to over-performing. 

Conclusion

A simple change can sometimes make a world of difference in a system's performance. It's essential to think outside the box and question existing architectural decisions. Interestingly, some of these simple solutions can be traced back to our college days, reminding us that even the most basic ideas and algorithms from our early education can still be powerful tools in our professional lives. 

While it may not always be possible to rewrite an entire system, a bit of ingenuity can often lead to significant improvements. Remember that good architecture is a roadmap, not a final state.

Comments

Post a Comment

Popular posts from this blog

Get-ChildItem vs Dir in PowerShell

Image
Batch as usual Recently I was in the need of modifying a huge amount of files across a network share. After installing Serviio media streaming service I noticed that it will crash "randomly". After checking the logs it was clear that ffmpeg was crashing when trying to open subtitle files that were not in the encoding indicated by the Serviio console. I needed a quick way to update all subtitle files to a common encoding, so I decided to convert all files to UTF8. Since I use a windows environment I went directly to PowerShell and wrote this.  Get-ChildItem -path .\ -filter *.srt -file  | ForEach-Object { (get-content $_.FullName) | Out-File $_.FullName -encoding utf8 } This worked almost right except for files with "[" or "]" (among other symbols) on the name or path. To solve it, just added the "-LiteralPath" switch to tell powershell not to consider any wildcards on the path name and just use it exactly as it is.  Get-ChildIt...
Read more

The case for a 4 week sprint

Image
Every time we talk about SCRUM, the length of the sprint is stated as a period of "two to four weeks", rarely, however, have I seen people going outside of the 2 weeks. This is not written on stone of course, and while 3 weeks might seems odd, there is a benefit to using a longer period for your sprints.  Image is taken from scrum.org Without any hard data to back that decision, the answer I get from asking about this decision seems to be related to "delivery speed", we assume that because we deliver value to the business every 2 weeks, we are in a more "agile" environment. Well, that might not be so true in every sense or for every project. Let's break some arguments: 1. Sprint length is not equal to the speed of delivery There is more to the speed of development and delivery than the timeframe of the delivery cycle, in a world with DevOps and full CI/CD implemented for your project, you could (should?) actually deliver features every single day all t...
Read more

NODE.js fs.readFile and the BOM marker

Working on a ReactJS project, got a small glitch while trying to read a JSON file and parse it as a javascript object. getDataObject() { var dataString = fs.readFileSync("./json/data.json", "utf8"); return JSON.parse(dataString); } output:   SyntaxError: Unexpected token   After checking, double and triple checking again that the file was correct I went deeper and realized that "fs.readFileSync" was returning me the BOM for the UTF file at the beginning of the string. So we just need to strip it out. getDataObject() { var dataString = fs.readFileSync("./json/data.json", "utf8"); return JSON.parse(dataString.replace(/^\uFEFF/, "")); } I then checked and there are "packages" for this, however I don't think is proper to just import a new dependency just to fix something this small. The big issue here is why is this considered correct. The BOM is not ...
Read more