I recently wrote some tests that ended up being flakey. After some investigation, I realized that the objects I was getting back from my Postgres database weren’t always in the order that I was expecting them to be.
I was using a single update or insert statement to create several records at the same time and then querying that same table. My initial experiments had indicated that the objects were always being returned in the same order that I passed them into the insert/update statement, but as I grew the number of tests using that assumption and ran them dozens of times, it became apparent that this assumption is flawed.
It appears that Postgres by default generally saves the objects in the order they are passed in the insert/update statement, and that it generally returns them in that same order, but sometimes it doesn’t.
I haven’t pinned down whether the breakdown is in saving, the retrieving, or both. My money is that it’s on the saving side of things, but it doesn’t actually matter for my purposes. For now, I’ve been updating these tests to return in a given order, ordering by something other than the created_at timestamp.
A good learning for me there about not leaving anything to chance when it comes to a test. Testing…one of the few places where being more explicit is better.