This year I found myself reflecting on one of the hardest research software projects I've worked on. The difficulty wasn't a hard algorithm or an unsolved technical problem. It was more like driving a manual car while someone keeps flooring the clutch: the engine roars, everyone can hear the effort, but the car barely moves. The team was working hard, but the project lacked alignment. Different stakeholders had different expectations of the destination, so we were pushing forward without ever fully engaging the gear.

Around the same time, I was working through the Google Project Management Certificate, mostly to broaden my perspective. I expected it to feel obvious. Instead, it gave me a framework for understanding what I had just experienced. Mapping the project onto the standard lifecycle (Initiation, Planning, Execution, Monitoring and Controlling, Closing), I noticed something uncomfortable: some phases happened almost by accident, while others barely happened at all.

That led me to a question: maybe research software projects do not struggle only because the problems are hard. Maybe we also lack a project lifecycle designed for the way research actually works. This is just one RSE's perspective, but it is a pattern I cannot unsee.

Mapping the Lifecycle to Research Software

Looking at the standard lifecycle through an RSE lens, the gaps become hard to ignore.

Initiation and Planning

In theory, initiation defines the problem, identifies stakeholders, and establishes what success looks like. In research software, that often gets replaced by the grant proposal. But proposals are written to win funding, not to serve as execution roadmaps. They describe a vision, often before the technical approach is fully understood.

Sometimes there is no proposal to rely on. The project starts with "make it work" or "make it better," and everyone fills in the blanks differently. Without alignment on the problem, it becomes difficult to agree on what is being built, or what "done" means.

Execution

Execution usually starts immediately because coding feels like progress. The pressure is almost always to start building, not to spend more time aligning expectations.

Monitoring and Controlling

This phase is more subtle. On paper, we were not ignoring it. We often used Agile practices where appropriate. The issue was not the framework; it was the capacity to maintain it once the project became busy.

Standups and sprint planning kept work moving, but they did not answer deeper questions: Is the software maintainable? Is it reproducible? Are we still solving the right problem?

Quality became whatever nobody had complained about yet. Risks were handled reactively, usually after they became problems, with little buffer to respond intentionally.

Closing

Closing rarely looks like the textbook version. Research software projects often end because a grant expires, funding runs out, or a demo is needed for a deadline, not because the work reached a clear definition of done. The final weeks become a scramble to make whatever exists presentable. Eventually, the software ships, the paper gets published, the demo goes well, but the project never really closes. It feels like a spacecraft that launches successfully but never completes its mission, like Beagle 2, lost after reaching Mars. The launch gets celebrated. The destination never does.

Successful by Accident

At this point, it is fair to ask: if these gaps are so damaging, why do so many research software projects still succeed?

My guess is that incomplete lifecycle management does not always look like failure. Something else fills the gap. Looking back, I see three common patterns.

1. Stakeholders fill the missing phases

Some projects work because stakeholders have already done much of the project management before the RSE team arrives. The problem is clear, requirements are aligned, and success criteria are understood. In these cases, the lifecycle still exists. It just happened before the engineers entered the room.

2. Execution masks the missing framework

This is the most dangerous category because it looks like success.

I know my own strengths. I am not the most technically brilliant programmer in the room, but I am effective because I execute quickly. I can move from ambiguity to implementation, adapt quickly, and find practical solutions. Those skills matter in research software. But they can also hide missing structures. A team can keep pressing the accelerator while the car is in the wrong gear. The engine gets louder, everyone feels the effort, but the destination does not get closer. Speed is not the same as direction. With the right structure, less effort can produce more impact because the team spends less time solving the wrong problems.

3. Long-running projects build the lifecycle through experience

Some mature projects appear to have strong project management, not because they started with a formal framework, but because they developed one over time.

Scientific committees create alignment. Years of iteration create development practices. Milestones, publications, and demonstrations become natural checkpoints. The lifecycle exists. It just evolved instead of being intentionally designed.

Why Research Software Needs Its Own Playbook

Why not just adopt existing project management frameworks?

The challenge is that many frameworks assume a stable scope, defined deliverables, and stakeholders who know requirements upfront. Research rarely works that way: a "requirement" is often a hypothesis. Success criteria change as findings emerge. Adapting to new knowledge is not poor planning; it is the nature of research.

In addition, research software also operates under different constraints. Dedicated project managers who understand the technical aspects are uncommon. RSEs often support multiple projects with very different needs, which makes estimation and planning hard. The rhythm is different too: grant cycles, conferences, student turnover, and changing scientific priorities create a project environment that traditional product models do not fully capture.

Closing the Gap

Closing this gap doesn't take an entire industry playbook. Nothing below is new; in fact every one of these habits already happens somewhere, in some project, some of the time. A PI writes down expectations once in a while. A team pauses to check quality after a bad bug. Someone remembers to write a technical report before they move on. The problem isn't that these practices don't exist, it's that they exist by chance, tied to whichever person happened to care that time.

What's missing is the next two steps: advocating for them as standard practice, then formalizing and enforcing them so they don't depend on who's in the room.

Some starting points

  • Align stakeholders early. A lightweight stakeholder check can clarify who is involved, who makes decisions, and what each person is optimizing for. A PI may care about a high-impact publication. A staff engineer may care about maintainability. A graduate student may need a thesis outcome on a deadline. None of these goals are wrong, but they are not always the same.
  • Create a lightweight project charter. Not a contract, but a shared record of the problem, current success criteria, decision ownership, and assumptions. It preserves the reasoning behind the code.
  • Plan for change. Research will change. Requirements will evolve, experiments will fail, and new findings will reshape priorities. Teams should revisit assumptions regularly, manage risks proactively, and decide whether to avoid, mitigate, accept, or resolve them before they become emergencies.
  • Protect quality. Code review, reproducibility checks, and architecture discussions cannot depend on spare time. Spare time rarely arrives.
  • Close the loop. Academia already has closing rituals: papers, talks, posters, and reports. Research software should treat releases, technical reports, demos, and retrospectives the same way.

The Reframe

None of this argues for importing corporate project management into research. It argues for something lighter: an approach that expects uncertainty, takes planning seriously, and treats quality and closing as real parts of the lifecycle, not afterthoughts. The goal is to keep the gear engaged, so effort turns into progress instead of noise.

I don't have the answer here. What's above is one RSE's observations and a few ideas worth exploring, not a methodology. But naming the gap is the first step. The next one is for the community to experiment, compare notes, and build practices that fit how research software actually gets made.

Here's a fact that gives me some hope: the Agile Manifesto wasn't handed down by a standards body. It came out of a group of software practitioners who gathered at a ski lodge in Snowbird, Utah, in 2001, frustrated with how software was being built and looking to compare notes on what actually worked. That's usually how new practice emerges, from people closest to the problem, experimenting toward something better. Maybe the next evolution in research software practice starts the same way: curiosity, trial and error, and everything the RSE community has already learned the hard way.