A messy mental map unfolds in my head as I think through the timing and alignment of a Feature Flag rolling into Production ahead of the feature itself and how both need to catch the Release Train.
No wonder someone coined the metaphor of a “train” as I envision pieces finally arriving at the end destination station called Production.
Features, flags and database changes often move on different timelines and yet still need to arrive in Production in coordination, often involving multiple teams with different sprint cadences.
The moving parts eventually come together to not just roll out a feature but to “stand up” the entirety of the feature in Production.
A feature built in code is like a car on the train.
Database changes are like train tracks running underground, laying boundaries for the data beforehand.
A feature flag is like a traffic signal, gating the code until all the parts arrive at the station.
Multiple teams are at work, each with their own Sprint cadence and yet ultimately the code, the database changes and the flag all need to catch the Release Train. A flag flipped “on” when the time is right, a feature unfurling into Production, database and configuration settings lined up correctly — it is a grand orchestration.
Testing isn’t always about visible user interface windows and pop-up messages. The invisible is sometimes the most interesting to test and probably why I can find bugs away from the keyboard at times.
Mental modeling.
Issues are not always what you can see physically but what you can see mentally as you draw on a whiteboard, a notebook or inside your own internal white space.
Thinking about automation, I think I’ve found a solution and then immediately realize a possible roadblock to what I thought might work.
Existing automation that works before this new feature could be re-used if there is a rollback or if the flag is turned off.
But as I mark up my notebook, it occurs to me that the database changes themselves might block the automation.
The existing automation may eventually break, but that break might not happen on a visible software release date. It could happen later when another team rolls in database changes. I’m not sure. There’s probably an interesting discussion there, the complexity of rollouts and the alignment of more than one team’s work.
When more than one team is building and making changes to a single product, each team has its own sprint cadence and pressure to move code onto the Release Train. Knowing where the code actually exists — in a test environment or in Production — takes coordination and oversight. That coordination is beyond my current role as one team’s tester but not beyond the kind of operational awareness testers begin to develop over time.
Earlier this week a developer was insistent that they had fixed a bug, reliable and rock solid work, so another tester and I weren’t calling it a bug but it was clear the fix wasn’t in the test environment. The developer had to dig in to find if her code had been included in the deploy and when she found out her code had made the release, all three of us were puzzled.
A bit later the developer mentioned another software program built by yet another team but essential to the application we were testing. She’d discovered the other team’s code changes hadn’t made the release. I think back to the train analogy and see disconnected train cars in my mental mind map. The puzzle pieces have to move in coordination or issues begin to surface.
The longer I test, the less I think only about features and the more I think about movement. Timing. Coordination. Dependencies. What arrives where and when. What happens if one piece rolls forward and another does not.
Somewhere along the way my testing stopped being only about what I could click on a screen. As my testing chops grew, so did the need to think beyond the visible and draw mental models of systems in motion.
The UI is simply the visible arrival point.
Underneath are feature flags, rollout timing, database changes, deployment coordination, environments drifting apart and multiple teams trying to move software forward together without disconnecting the train cars along the way.
Maybe this is why exploratory testing still holds my attention after all these years. The software itself matters, of course, but so does the operational choreography surrounding it. The invisible movement underneath before anything becomes visible to the end user.
Sometimes the problems reveal themselves while thinking through the alignment of all the moving parts.