An Intro to PARTNER’s Validated Measures
One of the things that makes PARTNER such a special tool for organizational network analysis are the metrics it uses to measure collaboration. Through a two-year process of interviews, surveys & focus groups, our team studied what mattered most when it comes to measuring networks. The result were a series of peer-reviewed papers and a set of scientifically-validated metrics that form the core of PARTNER. Through our experience employing them in the field, we’ve further improved them until we finally felt confident they capture what matters from an evaluation standpoint.
An Introduction to the PARTNER Platform Metrics
Here’s an overview of the metrics our PARTNER survey collects, along with a definition and explanation of how it matters in a network context. Want to learn how to analyze these steps using a network science lens? Click here to read about our learning lab opportunities, including webinars, events and an online course!
Network Structure Metrics
Network structure metrics concern the way that your partners are interconnected. There are innumerable ways to structure a network. You should strive to build the least number of connections required to reach your shared goals, rather than building as many relationships as possible. Here are our five main network structure metrics.
Density Definition:
Percentage of ties present in the network in relation to the total number of possible ties in the entire network, if everyone was connected to everyone else.
Density Interpretation:
An indicator of interconnectedness. Holding the number of nodes in a network constant, an increase in connections between the nodes means an increase in density. To achieve a 100% density score, every member of the network would have to be connected to every other member.
Based on VNL’s database of over 4,000 inter-organizational networks that have been analyzed using the PARTNER Platform, we use the following scale to assess density scores:
- Low: <20%
- Moderate: 20-60%
- High: >60%
Centralization Definition:
A measure of the extent to which a network is dominated by one or a few very central hub-like nodes. The lower the centralization score, the more similar the members are in terms of their number of connections to others
Centralization Interpretation
In a highly centralized network, these central hubs represent single points of failure which, if removed or damaged, quickly fragment the network into unconnected sub-networks. A less centralized network has fewer potential points of failure and exhibits greater resilience, since many nodes or links can fail while allowing the remaining nodes to still reach each other over other network paths. A lower centralization score may also indicate that activity and popularity of nodes in the network is relatively equally distributed.
Based on VNL’s database of over 4,000 inter-organizational networks that have been analyzed using the PARTNER Platform, we use the following scale to assess density scores:
- Low: <20%
- Moderate: 20-60%
- High: >60%
Degree Centrality Definition:
The total number of edges connected to a node (ingoing and outgoing). Average degree measures average number of edges reported for each node in a network.
Degree Centrality Interpretation:
A measure of a node’s activity or popularity in a network. A node with a high degree centrality score may be very active in forming relationships with other nodes, and/or may be a popular partner with which to connect. Such a node may act as an “opinion leader” or “influencer” in the network, and may possess more information than others about the network simply because of its greater number of connections to others.
Degree centrality scores are best judged on a scale relative to the range of other scores in a particular network. The maximum degree centrality score is always the total number of nodes in a network, minus one (because a node cannot be connected to itself). The minimum score is always zero.
Closeness Centrality Definition:
The average distance of a node from all other members of the network, in terms of number of edges between each member.
Closeness Centrality Interpretation:
A high score (close to 1) indicates members who have the shortest ‘distance’ between all other members. Members with high closeness centrality are considered central because they can most easily reach other members of the network. A high closeness centrality score indicates a network member who may be able to wanted to quickly spread news or best practices within the network.
Closeness centrality scores are best judged on a scale relative to the range of other scores in a particular network, and range from 0 to 1.
Betweenness Centrality Definition:
The frequency with which a node lies on the shortest path connecting others in the network.
Betweenness Centrality Interpretation:
Nodes with high betweenness centrality scores are sometimes referred to as “boundary spanners” because of their ability to span boundaries or distances in a network and connect otherwise unconnected nodes or groups of nodes. A person can have a relatively low degree centrality score but a high betweenness centrality score if, for example, that person does not have many friends, but the friends he or she does have are from otherwise different, disconnected cliques.
Betweenness centrality scores are best judged on a scale relative to the range of other scores in a particular network, and range from 0 to 1.
Network Trust Metrics
Trust is the foundation of effective networks and community partnerships. However asking about it openly rarely gets you valid responses. We use a three-indicator measure for trust that’s been validated scientifically using interviews and surveys. Here are the three indicators and overall trust metric.
Overall Trust Definition:
A PARTNER measure of how much members trust one another by capturing members’ perceptions of others along three dimensions: reliability, mission alignment, and openness to discussion. The overall trust score reflects the average of the scores along the three dimensions. Assessed at both the whole network and individual node levels.
Overall Trust Interpretation:
Survey respondents rate each dimension of trust on a scale of 1 to 4 (1=not at all, 2=a small amount, 3=a fair amount, 4=a great deal). Scores of at least 3 out of 4, or 75%, are considered good scores. At the network level, if a network receives a 100% trust score (4/4), it would mean that all members reported the highest levels of trust of one another. If any dimension of trust receives a score under 3.0, this may indicate an area of growth for the organization. Each of the three dimensions of this combined score are considered equally important.
Trust Score: Reliability Definition:
The organization follows through on commitments.
See Overall Trust for Interpretation Notes
Trust Score: Mission Alignment Definition:
The organization is perceived as being committed to the shared mission or goal of the network.
See Overall Trust for Interpretation Notes.
Trust Score: Openness to Discussion Definition
The organization is willing to engage in frank, open and civil discussion (especially when disagreement exists). The organization is willing to consider a variety of viewpoints and talk together, rather than at each other.
See Overall Trust for Interpretation Notes.
Network Value Metrics
Organizations collaborate with each other because they find value in partnership. However there are various types of value, including power and influence, time and energy, and resource contribution. We use these factors to create an overall value score for each partnership in the network.
Overall Value Definition:
A PARTNER measure of how much members value one another by capturing members’ perceptions along three dimensions: power/influence, level of involvement, and resource contribution. The overall value score reflects the average of the scores along the three dimensions. Assessed at both the whole network and individual node levels.
Overall Value Interpretation:
Survey respondents rate each dimension of value on a scale of 1 to 4 (1=not at all, 2=a small amount, 3=a fair amount, 4=a great deal). Scores of at least 3 out of 4, or 75%, are considered good scores. At the network level, if a network receives a 100% value score (4/4), it would mean that all members reported the highest levels of value of one another. If any dimension of value receives a score under 3.0, this may indicate an area of growth for the organization. Each of the three dimensions of this combined score are considered equally important.
Value Score: Power & Influence Definition:
The organization holds a prominent position in the community by being powerful, having influence, having success as a change agent, and showing leadership.
See Overall Value for Interpretation Notes.
Value Score: Level of Involvement Definition:
The organization is committed to and actively involved in achieving network goals. The organization gets things done.
See Overall Value for Interpretation Notes.
Value Score: Resource Contribution Definition:
The organization contributes resources like funding, information, staff time, meeting space, etc.
See Overall Value for Interpretation Notes.
Network Intensity Metrics
Not all partnerships are the same, as they range in intensity from simple awareness to full integration of their activities and efforts. Here is our network intensity metric and scale.
Relationship Intensity Definition:
A PARTNER scale that measures relationship intensity. The question prompt asks, “What is your organization’s dominant way of interacting with this organization?”
Relationship Intensity Interpretation:
Can be used as a measure of tie “strength.” Scale, from low intensity to high: awareness, cooperation, coordination, integration.
Intensity Score – Awareness Definition:
Having basic awareness of what a relational partner does (e.g. understanding of services offered, resources available, mission/goals).
Intensity Score – Awareness Interpretation
A network with relatively high percentages of awareness relationships may be newly formed, or may be attempting to encourage relationships among organizations that typically don’t interact. To capitalize on collaborative advantage, networks should attempt to convert many of its awareness relationships to those of higher intensity. However, don’t underestimate the strength of weak ties. Basic awareness can lead to the discovery of new resources, contacts, etc., for an organization.
Intensity Score – Cooperation Definition
Having awareness of a relational partner AND informally exchanging information, attending meetings together, and sharing in-kind resources.
Intensity Score – Cooperation Interpretation
A goal for most networks is to have high percentages of cooperative and coordinated relationships.
Intensity Score – Coordination Definition
Cooperating with a relational partner, AND synchronizing activities for the mutual benefit of services and/or programs (e.g., sharing proprietary data, timing events/activities together).
Intensity Score – Coordination Interpretation
A goal for most networks is to have high percentages of cooperative and coordinated relationships. In networks in which mission alignment is particularly important, coordinated relationships may be desirable.
Intensity Score – Integration Definition
Coordinating with a relational partner, AND entering into mutual, binding relationships that support work in related content areas (e.g., through contracts, grants, MOUs, pooled funding with collective implementation).
Intensity Score – Integration Interpretation
When thinking about relational integration, we tend to revert to the conventional wisdom that “more is better,” and of course a highly integrated relationship may yield many benefits. Keep in mind, however, that there are also drawbacks to high integration: the greater the degree of integration, the more time and effort the relationship likely requires. A network with relatively high percentages of integrated relationships may be at risk for member burnout and eventual disengagement.
About the Author: Alex Derr, M.P.A.
Director of Marketing & Communications
Alex joined VNL in 2017, originally supporting our events. He now helps manages our communications and marketing strategy and content development work. Alex creates blogs, infographics, reports, and other content while managing our web and social media presence. He also runs our email marketing campaigns, tracks analytics, and conducts market research to drive our strategy. He supports our entire team with copywriting, graphic design and research, and helps with events, webinars, demos, and other online learning. When he isn’t at work Alex spends his time climbing 14ers (30 done, 28 to go!) and blogging on his own website, The Next Summit Blog.
