Process Change

Process. The word itself can elicit groans from engineers. For many, it feels like unnecessary bureaucracy, a stifling of creativity, and a distraction from actually building things. But as an engineering leader with over two decades of experience, I’ve seen firsthand that process isn't the enemy – poorly implemented or unnecessary process is. And the biggest challenge isn’t having a process, it’s changing one.

This post is for engineering leaders grappling with process change. It’s about how to introduce, adapt, or overhaul processes without triggering a revolt, crippling velocity, or – worst of all – losing the trust of your team. It’s a tightrope walk, requiring empathy, transparency, and a healthy dose of pragmatism.

The Allure & The Peril of Process

Let's acknowledge the inherent tension. On one hand, a well-defined process provides structure, predictability, and a shared understanding of how work gets done. It allows for better estimation, risk management, and scalability. As Kaur and Sengupta (2011) highlight, establishing clear roles and processes before success is crucial (p. 2.2-2.8). Think of it as “workflows as code” - free to run, and powerful when executed correctly.

However, Kaur and Sengupta’s research also points to a critical flaw: teams often become paralyzed by process when the focus shifts to the process itself rather than the desired outcome. I've seen teams become consumed by documentation requirements, endlessly debating minor changes in meetings, and ultimately delivering less value. There's a cynical truth in the quote, attributed to Tom DeMarco in Slack Time, "They are just something we tell customers because they are appalled when we admit that our software is developed in a chaotic and unprofessional manner." We sometimes adopt process for appearances instead of genuine improvement.

Why Process Change Fails & A Framework for Success

Before diving into how to change process, let’s pinpoint the common pitfalls. These aren’t roadblocks to avoid after you’ve implemented a framework, but rather problems you should actively address within your process improvement strategy:

• Top-Down Dictates: Imposing a new process without involving the team is a recipe for disaster. Imagine a lead engineer mandates a new code review process without consulting the team. Engineers are problem solvers; they’ll instinctively identify flaws and resist something forced upon them.
• Lack of Clear "Why": If the rationale behind the change isn't compelling and clearly communicated, it will be perceived as arbitrary. "We're doing it because that's how things are done now" rarely motivates.
• Overly Prescriptive Specifications: Specifying how to solve a problem, instead of what problem needs solving, stifles innovation and disengages engineers.
• Ignoring Existing Pain Points: Introducing a new process on top of existing frustrations won’t fix anything. Address underlying issues before layering on more complexity.
• No Feedback Loop: Rolling out a change and then disappearing is a mistake. Continuous monitoring and adaptation are essential.

To avoid these pitfalls, consider this framework:

1. Diagnose, Don't Prescribe: Before proposing any changes, understand why things are the way they are. Talk to your team. What are their biggest pain points? What's slowing them down? What works well? This isn't about finding fault; it’s about identifying opportunities for improvement.

2. Co-Create the Solution: Involve your team in designing the new process. This doesn’t mean everyone gets a veto, but it does mean their input is valued.

3. Timeboxed Exploration & Feasibility Studies: Introduce "timeboxed explorations" during the planning phase. Give engineers dedicated time—typically one to two sprints—to investigate the feasibility of proposed solutions and identify potential roadblocks. The output should be a clear assessment of the benefits, costs, and risks associated with each option, along with concrete recommendations.

4. Focus on Outcomes, Not Just Process: Clearly define the desired outcomes of the change. What problems are you trying to solve? How will you measure success? The process is a means to an end, not the end itself.

5. Iterate & Adapt: No process is perfect. Establish a feedback loop and be prepared to adapt the process based on real-world experience. Regular retrospectives are invaluable for identifying areas for improvement.

6. Transparency & Communication: Keep your team informed throughout the entire process. Explain the rationale behind the changes, solicit their feedback, and be honest about any challenges.

A Word of Caution: The "Just Work It Out" Trap

While autonomy is important, simply telling engineers "you just work it out" often leads to chaos and instability. A lack of structure can be just as detrimental as excessive bureaucracy. This approach neglects the benefits of shared understanding, consistent quality, and long-term maintainability. The sweet spot lies in finding the right balance – a process that provides enough guidance and structure without stifling creativity or hindering velocity. It’s about acknowledging that a lack of process, while seemingly liberating, can introduce hidden costs in the form of technical debt, inconsistent results, and knowledge silos.

Final Thoughts

Process change is rarely easy. It requires a delicate balance of leadership, empathy, and pragmatism. Remember that your team isn’t resisting change; they’re resisting poorly implemented change. By involving them in the process, focusing on outcomes, and being open to feedback, you can navigate this tightrope walk and create a more efficient, effective, and engaged engineering team.

What's one small change you can make this week to improve your team's process? Consider which existing process in your organization could benefit from a timeboxed exploration.

Illustration Idea: A simple diagram illustrating the “Process Spectrum” – ranging from “Chaotic/No Process” to “Rigid/Bureaucratic” with an “Optimal Zone” highlighting the desired balance.

Further Reading:

• Kaur, P., & Sengupta, D. (2011). A framework for agile project management using lean principles. International Journal of Lean Six Sigma, 2(1), 83–106.
• DeMarco, T. (1982). Slack Time. Prentice Hall.