Evaluating a Gates Foundation Community of Practice to Drive Impact

The Networks for School Improvement Portfolio.

The K-12 team at the Bill & Melinda Gates Foundation aims to significantly increase the number of Black and Latino students, and students experiencing poverty who earn a diploma, enroll in a postsecondary institution, and are on track in their first year to obtain a credential with labor-market value.

Networks for School Improvement are built upon a central hypothesis: When school teams are able to use continuous improvement approaches to achieve better outcomes for students and learn from each other with the help of a supporting intermediary organization, they will advance high school graduation and college readiness and success.

Through three competitive Request for Proposal (RFP) processes, 30 organizations in over 20 states were selected for this portfolio, and all were invited to participate in the Networks for School Improvement Community of Practice (NSI CoP). Within the NSI CoP, partners support each other to share best practices and innovations that improve learning outcomes for their own school networks.

The Foundation prioritized learning how the network was functioning and impacting outcomes.

With the local networks assembled and the Community of Practice up and running, the Gates Foundation wanted visibility and understanding into how this “network of networks” was making impact on outcomes. Specifically, they sought to learn:

  • What are participants gaining from being part of this network of intermediaries?
  • What relationships among NSI intermediary organizations are forming, and why are they motivated to connect?
  • What information is being shared among members, and how are members leveraging connections to access resources and advance their outcome goals?
  • Is the peer-to-peer approach of the NSI CoP working, and how could the Gates Foundation improve the NSI CoP?

Developing a plan, collecting, analyzing and visualizing the results.

VNL began working with the K-12 team at the Bill & Melinda Gates Foundation to design an evaluation plan that would provide continued insights into and measure impact over time of the Community of Practice by asking members of the group about their perceptions of one another, along with the value they’ve received from collaborating through this community. We worked with the foundation team to prioritize the questions they wanted to ask, used our validated methodology to boost response rates, and generated the data and insight needed to assess the work of the NSI CoP and improve outcomes for students. Once the surveys were collected, we then mapped and analyzed data to start translating their findings into actionable insights.

Translating the results into actionable network insights for growth.

The visualization of the network helped the foundation test their assumptions about their Community of Practice and its participants in terms of centrality and influence. While some of the findings were unsurprising, it also helped them identify partners and influence they may have missed without some kind of network analysis. With a bird’s eye view of their network’s structure, they better understand how their partners interact with each other, share information, and leverage value from peers in the community. Two findings stood out in validating their network approach:

  • More than half of respondents stated their home organization developed or improved programs or services it delivers as a result of participation in the NSI CoP.
  • CoP participants overall demonstrated their interest in sustaining the network, illustrated by two-thirds of respondents stating they were either somewhat likely or very likely to stay engaged with their partners even without funding.

The network data collected via PARTNER, the Platform to Analyze, Record and Track Networks to Enhance Relationship, is helping the team at the Gates Foundation adjust plans for the NSI CoP, and find opportunities to strengthen its impact on K-12 education for Black and Latino students and students experiencing poverty.

More Network Leadership Case Studies