Timing Your Network Evaluation: How Often Should You Survey Your Partners?
By Marissa Baron & Jenny Lawlor
When working with clients and community partners, the VNL team is frequently asked two questions:
- When is it the right time to conduct a network evaluation survey?
- When is the right time to conduct a follow-up or second round of network data collection?
Our answer is: There is no single correct answer or prescribed frequency for collecting network data for evaluation and continuous improvement. Rather, it is important to consider the context of your network and identify indicators that may signal an effective, appropriate time to collect network data.
Tools To Identify Opportunities for Engaging in Network Analysis
Many organizations across the nonprofit, public, and philanthropic sectors use a theory of change criteria or methodology to plan, implement, and evaluate social change that aligns with their organizational mission and vision. A logic model or a theory of change is one tool that can be leveraged to identify critical distinctions between desired and actual outcomes. Establishing a baseline network analysis in alignment with your organization’s theory of change can help identify distinctions between short-, intermediate, and long-term outcomes. Additionally, data collection may be helpful when assessing activities and outcomes focused on collaboration or collective work established through the theory of change.
Another tool organizations may consider in identifying when to conduct a survey is a network sustainability plan. A sustainability plan serves as a roadmap toward achieving long-term collective goals, particularly documenting strategies to maintain and support existing programs, activities, and partnerships across the network. Similar to the theory of change, establishing a baseline survey in alignment with your network sustainability plan can help identify critical moments or connections across your network to identify how it can support sustainability. Further, notable shifts in your sustainability plan may indicate an effective time to re-engage in data collection.
Indications That It May Be a Good Time to Collect Network Data
While the theory of change and sustainability plan may be two effective tools to note beneficial points in time to engage in data collection, both of these approaches still require an assessment of economic, environmental, and social changes–both internal and external to the network. Several indicators that may elicit an effective opportunity to engage in network analysis include:
1. Change in the Network
When shifts occur that are likely to highlight noticeable changes in the relationship structure, attitudes, or content of the network’s efforts, it may be time to conduct a survey. Several shifts that may indicate such a change include: growth or change in membership across the network; change in organizational or network-wide capacity or resources; and change in the mission or goals of an organization within the network or network-wide (e.g., organizational strategic planning).
2. Change in the Context
In addition to substantive changes across the network over time, the surrounding environment and context in which the work of the network is done will shift. Such changes in the community, whether environmental changes in the long-term or significant shocks to the system (e.g., COVID-19 pandemic), can create drastic shifts in the network. Additional systemic changes such as a policy change or political shift may influence organizations within the network or the network-at-large to consider an opportunity for data collection.
3. Change in Reporting or Evaluation Requirements
Reporting and evaluation requirements often create an opportunity for network data collection. Funders or fiscal sponsors often set requirements for the type and frequency of data collection. When reporting or evaluation requirements focus on collaborative activities, relationship-building, or how groups leverage their partnerships, it can be useful to collect network data. Networks may consider reporting or evaluation requirements in tandem with other indicators on this list to optimize their data collection efforts.
4. Baseline Measurement of a New Network
When a new network is forming, network members often have pre-existing relationships (Kegler, Rigler, & Honeycutt, 2010). Conducting a survey of new network members can illuminate the existing topography of collaboration, highlighting key members who can disseminate information, areas of strong collaboration, and opportunities for deepening or creating relationships to achieve shared goals. This data can inform strategy for network development and can be used as a baseline to identify progress with future network survey collections.
5. The Network Moves to a New Stage of Development
Networks often move through different stages of development as they work toward change goals (Granner & Sharpe, 2004; Kegler & Swan, 2011). As networks continue to develop, moving beyond their initial formation towards a stable and sustained effort, their structure and function will change. Recognizing changes in group development can indicate that it is a good time to capture network data. This can indicate how the network’s structure supports the current goals and activities, while identifying opportunities for adjustment.
How can you maximize success when you need to conduct a follow-up survey with a short turn-around?
Sometimes it makes sense to use follow-up network surveys in a short time period based on the indicators above (e.g., shortly after conducting a baseline survey, an influx of resources makes it possible for more members to join the network). To avoid overburdening participants with multiple surveys, consider using a short version of the VNL standard survey, asking 5-10 questions rather than the typical 18-20 question survey.
Another option is to constrain the surveyed groups. For example, you might only survey new network members to capture their engagement with the whole network. Additionally, If a sub-group or sub-network has experienced significant change, another option is focusing a survey effort only on that sub-group.
Timing Your Network Data Collection: Now You Know!
In sum, there is no single timeline that can dictate if it is an appropriate time to capture network data. Using network planning documents, like a theory of change or sustainability plan, can suggest initial opportunities for collecting network data. Other indicators can also shape the timing for collecting network data, including changes in the network or community, changes in evaluation or reporting requirements, the need for a baseline measure of a new network, and changes in a network’s developmental stage.
When considering data collection timing, scheduling sufficient time between surveys can support high levels of participation. If a network needs to survey its members frequently, taking steps to minimize burden can ensure high participation and quality data. Of course, VNL is always happy to help identify if it is the right time to capture network data. Please reach out with questions or inquiries about your network!
References and Additional Reading
Granner, M. L., & Sharpe, P. A. (2004). Evaluating community coalition characteristics and functioning: A summary of measurement tools. Health Education Research, 19(5), 514–532. https://doi.org/10.1093/her/cyg056
Kegler, M.C., Rigler, J. & Honeycutt, S. (2010). How does community context influence coalitions in the formation stage? A multiple case study based on the Community Coalition Action Theory. BMC Public Health, 10(90). https://doi.org/10.1186/1471-2458-10-90
Kegler, M.C. & Swan, D.W. (2011). An initial attempt at operationalizing and testing the community coalition action theory. Health education & Behavior, 38(3), 261-270. DOI: 10.1177/109019811037285
More News from Visible Network Labs
VNL Wins the Prime Health Innovation Challenge for PARTNERme
