This week I had a really interesting problem to tackle: one simple script of mine behaved not as expected. And it took me couple hours of analysis for a simple ~30 line script to find the cause.
The script on itself its quite simple one – we have 2 numbers and 2 dates, one of them is in the past, another is in the future. We need to calculate current value from current date and 4 other values. To complicate the problem, we have to randomize things a bit by adding something (always positive value compared to last one) up to the current value so we do not get a straight line from starting point to the finish. There are some other conditions that impact calculation.
Ending coordinates are constant during whole execution of the script. Starting coordinates would be constant if we had no randomization as well. So, we store starting coordinates in separate .php file that is loaded in main script. Then new values are calculated, checked if they are not out of bounds and written down as new starting values to the same config file.
Testing provided no problem with the script, however in work environment sometimes counter went back in time. And no one could find the issue. The added value was always positive. The configuration file was saved. There was single configuration file used and no other programs overwrote it. Everything seemed ok and script worked incorrectly sometimes.
The problem was server load and disk space. During high loads on server, script execution was not atomic as expected, and file writes were slow too. So when 2 scripts executed at the same time, include file might be empty. Thus script would recalculate next value without having the data about previous “randomness” and it would be smaller than it should be.
The moral is that I should have expected similar problems. It is easy to forget concurrent environment of the web when programming or analyzing simple scripts.