Build the Agile Guideline Development Program

Last year, in early 2024, my team began the process of scaling our new operations model for PICO-centric guideline development (aka question-centric model, QCM). Our work was guided by the following aims:

• Maintain or improve workflow efficiency

• Maximize the effectiveness of our human resources (staff, consultants, volunteers)

• Strategically prioritize clinical questions

  
  

Building on lessons from our successful QCM pilot and a detailed "value stream map" from 2023, we targeted key program elements essential for managing workloads and enhancing workflow.

Looking Back: 2023 Highlights

Over the past year, we refined our approach to prioritizing clinical questions, ensuring resources were directed toward the highest-impact areas. We incorporated new mental models to guide strategic decision-making in both project selection and collaborative efforts. As we progress, our team continues to fine-tune operational strategies, exploring innovative ways to manage our portfolio efficiently and effectively, while ensuring our program remains agile and our work has the greatest possible impact.

Below, I’ll share an update on how we’re developing a clearer strategy for managing our workloads, describe early successes of our QCM approach, and outline our current steps for improving the adaptability and focus of our program.

Prioritizing Our Portfolio for Impact

Our primary goal at the start of 2024 was to reshape our program to swiftly address stakeholder priorities and urgent clinical questions. However, the sheer volume of existing work significantly exceeded our capacity—by 5-10 times our ideal—extending our timelines to several years beyond target completion dates. Consequently, many projects had stalled, exceeding or nearing literature search expiration deadlines.

“You can do a lot, or you can move quickly, but you can’t do both.” - Frances Frei

To achieve effective scalability, we focused on two critical objectives: (1) creating capacity to complete active clinical questions swiftly, and (2) developing robust frameworks for systematically prioritizing questions from across our entire portfolio (i.e., identify our next project, from all possible projects). This required evaluating each project's potential impact and strategically allocating resources accordingly.

We initially asked each panel to pause less active or early-stage projects, placing them into a “queue” for later development, so they could more quickly finish the clinical questions that were most active and furthest along in their development. From the remaining work, we then determined which questions could be released independently, and which questions should be batched and their manuscripts released together. This shift brought our workload closer to 2-4 times our ideal capacity.

We then conducted a risk assessment of the program itself, to determine how our remaining workload impacted the average timeliness of projects and to estimate the required timeframe to complete development of the paused projects. By using the timeline and rough-order-of-magnitude (ROM) workload estimates generated with our value stream mapping (VSM), we concluded that our queue would likely require most of the remaining decade to complete. Moreover, any additional urgent projects would further strain the system, necessitating pauses on ongoing work. Crucially, we recognized the absence of a coherent, consistent framework for prioritizing high-impact clinical questions—a necessary foundation for workload management in our QCM operations model. 

By addressing our immediate capacity challenges through strategic prioritization, we not only reduced workload congestion but also set the stage for achieving meaningful milestones and further refining our operational approach.

Celebrating Milestones and Building for the Future

By mid-year, our efforts to begin scaling our new operations model bore fruit with the release of our inaugural QCM-style publications. These represented a pivotal step: adapting an existing guideline manuscript to align with our new templates and SOPs. Seeing these publications come to life—first on the web and later in print—marked the culmination of the team's dedication and collaboration as they transformed a conceptual framework into a tangible, impactful reality. An idea became an innovation, a new “technology”.

Following these successes, we refocused efforts on refining how we prioritize clinical questions across our portfolio. Recognizing the need to balance the technical requirements of guideline development with the values and needs of our clinical experts, my team began designing a new, question-level prioritization process that would allow us to rank-order high-priority projects based on their potential impact and urgency. We spent the late summer and early fall executing the prioritization process across six panels and 68 questions. 

Our next steps include generating a consolidated shortlist of high-priority questions across all panels, ranked with insights from members and volunteer leaders, and integrating our new question-level prioritization process with our existing topic-level prioritization. This integrated, cyclical workflow will guide both topic-level and question-level prioritization, ensuring targeted resource allocation.

Once fully implemented, this approach will allow us to manage our portfolio of guideline projects more strategically, allocating resources in a targeted manner and ensuring that our guideline development efforts deliver maximum value to our stakeholders.

Refining Our Strategy for Prioritization and Collaboration

“The essence of strategy is choosing what not to do.​” - Michael Porter

Throughout most of 2024, my primary focus was developing a structured framework to prioritize clinical questions effectively. The complexity and expanding range of possible questions underscored the importance of strategically selecting areas of greatest potential impact. 

Complicating things further, our guidelines have limited impact and value if they aren’t used by decision-makers. Due to the structural realities that many of our clinical experts practice within, patient-care decisions are often made by other specialists who refer to competing –and sometimes conflicting– guidelines developed by their own medical specialty societies. While our organization has a long history of collaborating on and endorsing guidelines developed by other organizations, these efforts were not explicitly considered as part of the strategy by which we could impact clinical practice.

The goal, then, was to create a framework that maximizes the impact of our clinical and methodological expertise by identifying the highest-impact clinical questions, while also embedding collaboration and endorsement opportunities into our operational strategy. The approach would blend practices from methodology frameworks with mental models used by business strategists and academicians. 

We started with the traditional framework of Competitive Advantage (1), described by Constantinos Markides as a theory and method by which businesses prosper in a crowded economic “marketplace” by offering unique products or services to a specified customer base. 

Who is the "customer"?

We began building our framework by identifying the “customer base” our program should target, and what “unique products” our portfolio should include: 

1. Who are our customers? 

2. What guidelines should we develop?

   
   

The target audience for our ideal portfolio revealed a wide variety of clinical specialists commonly involved in decision-making informed by our guidelines. However, as a non-profit entity with limited resources to maintain our over-burdened portfolio of projects, we couldn’t directly apply the ideas of Competitive Advantage by trying to out-compete other guideline developing organizations. Instead, we inverted the model and instead framed our “competitors” as a network of medical specialty societies that represented collaboration and endorsement opportunities.

Therefore, to complete our framework we integrated the mental model of Business Ecosystems (2) to clarify when to leverage our own methodological and clinical experts to lead guideline development, and when to contribute to externally led efforts:

3. How do we decide when to lead, support, or endorse guideline development to maximize impact and adoption? 

These three questions became the foundation of our new framework to further refine our selection and prioritization of projects and ensure our resources are allocated to high-impact clinical questions. Furthermore, by applying an ecosystem mindset, our portfolio strategy better aligns our organizational expertise and capabilities with the broader needs of the healthcare landscape.

Exploring Models for Organizational Design

“Every system is perfectly designed to get the result that it gets.” - Paul Batalden

As we work to finalize these updates in 2025, we’ve begun integrating our new question-level prioritization process with the existing topic-level process to create a singular, iterative system, and will be executing the processes this year. There’s still much to refine, and we’ll continue to experiment and build upon our learnings. We’ll also work to improve our original VSM estimates, collecting data on key milestones to develop more precise project budgets and resource allocation strategies and, ultimately, managing workloads to maintain volumes which maximize timeliness.

An ongoing priority is updating the operational strategies for our guideline development program: the policies and processes which define how we identify the highest-priority clinical questions and ensure their timely publication. The crux of this challenge is to clearly communicate the anticipated impacts on guideline timeliness and volume, and how the program will improve both of these key metrics over time.

As these policies and processes are finalized and implemented to form the foundations and systems which support and power a more agile guideline development program, we’re continuing to build toward the long-term goal of optimizing the organizational design and management of the program. 

Building a guideline development program?

However, designing a program that can quickly identify and address the most pressing clinical questions requires a model which balances decentralized autonomy (e.g. to panels of clinical experts) with centralized coordination (e.g. to an oversight committee) , ensuring that rapid responses to new challenges do not compromise overall strategic alignment. Furthermore, scaling a program which can respond dynamically to changing demand requires a model that ensures adaptability through structured decision-making at multiple levels.

Therefore, as we continue refining the design of our program, we’ll be exploring applications of the Viable System Model, an organizational framework developed in the early 1970s and most famously used in Chile to model their social economy. Based on living systems, the model is designed to help organizations adapt effectively in changing environments. It structures an organization into distinct but interconnected parts, each responsible for specific roles. By balancing autonomy (the ability of teams or units to act independently) with centralized coordination (ensuring alignment with overall goals), VSM helps maintain both agility and stability, allowing organizations to respond rapidly to new challenges without losing sight of their strategic objectives.

As we finalize the above policy and process changes over the next months, this model will serve as another tool by which we rethink and make adjustments to the design of our program. Applying the Viable System Model will help us refine our decision-making structures, allowing us to scale our work more effectively while maintaining a clear focus on high-priority clinical questions. In doing so, we aim to create a guideline development program that is not only more agile but also more capable of delivering timely, practice-changing recommendations.

References:

1. Markides C., All the Right Moves: A Guide to Crafting Breakthrough Strategy. Harvard Business Review Press. 1999.

2. Lewrick, M. Design Thinking for Business Growth: How to Design and Scale Business Models and Business Ecosystems. Wiley. 2022.