How Do We Know if Our Network is Effective?

May 29, 2014

Approaching Network Assessment

Using evaluation to learn about how well a network is serving its goals means asking and answering questions about what’s working in partnership with those involved, sharing what we’re learning so that others can benefit from our experience, adapting our network in response to lessons learned, and then asking new and better questions. This continuous assessment is based on a collaborative process, where ownership of insights and recommendations is shared and thereby motivates collective action and progress toward goals.

Shared ownership of results does not mean disregarding accountability concerns. If anything, accountability is increasingly important in a network context, where responsibility and action are decentralized, but the work needs to keep moving and everyone has their commitments to fulfill.

One of the greatest barriers to grantmaker investment in networks is showing near-term measurable returns. Investing in networks requires patience and a willingness to embrace emergent and unexpected outcomes in addition to the original target.

Although the number of funders investing in and experimenting with networks and network approaches is growing, much of the evidence about network effectiveness is qualitative or anecdotal. By developing an approach to assessing and learning about network impact that reflects the nature of networks, funders can set appropriate expectations for the return on investment.

“You just never know when the value of a network will become clear. This can be difficult for grantmakers that seek a linear return on investment. Yet as networks grow they build upon many small acts of relationship-building, problem-solving and knowledge-sharing…The key is patience: Networks may lie dormant for a while, but activate quickly when necessary.” – Roberto Cremonini, GivingData (formerly of Barr Foundation

Network Assessment Principles

Despite these challenges, we’re learning more and more about how to assess network impact. Although there is no easy formula, there is an emerging set of principles that can help inform network learning and impact assessment and, more generally, our understanding of efforts to change complex systems. These principles for assessing networks include: considering the context, assessing multiple pathways to impact and enabling ongoing learning and collaboration.

1. Consider the Context

Recognize the interplay and interconnection between the larger world and the network. Life happens. There are economic booms and busts. Newly elected leaders come into office. Networks are embedded in these changing contexts, where the context changes the network and the network changes the context.

Calibrate results against what might be expected at a given point in a network’s life cycle. Understanding the various life cycles helps create a shared understanding of the network’s current state, challenges and future potential. During the impact assessment process when assessing a network’s vibrancy and connectivity, compare this measurement with where the network is in its life cycle.

2. Assess Multiple Pathways

Focus on meaningful contribution toward impact rather than attribution. Given the complexity of networks and the systems in which they’re embedded, it’s rarely possible to attribute causality to a network in light of the many interrelated and constantly changing players and activities. In addition, many significant changes can’t be measured immediately or in quantitative terms, and what can be measured may not always be what’s most important. Instead, the focus should be on how network participants and projects are contributing toward long-term aspirations.

Look at indicators of impact at the multiple levels on which a network operates:

Interpersonal connectivity: What is the nature of relationships within the network? Is everyone connected who needs to be? What is the quality of these connections? Does the network effectively bridge and embrace differences? Is the network becoming more interconnected? What is the network’s reach?
Network formation: How healthy is the network along multiple dimensions: participation, form, leadership, capacity, and so on? Also, what products and services are the immediate results of network activity?
Field-level outcomes: What progress is the network making on achieving its intended social impact (e.g., a policy outcome, an improvement in public health, an increase in community prosperity)? How do we know?

Evolve the evaluation approach with the network. Because networks themselves are dynamic and always evolving, it’s impossible to fully determine the evaluation design in advance. It will likely shift as the network changes.

3. Contribute to Ongoing Learning

Assess often and early. Recognize that patterns of network activity may be sporadic and spread out over time, and adopt approaches to learning and evaluation that reflect this rhythm. Develop benchmarks and ways to assess interim progress so the network can adapt. Early stage and regular evaluation can also be a way to find things to celebrate and thereby increase momentum and commitment to the shared work.

Emphasize learning over near-term judgment, given the long time horizon for many networks. Taking stock of what a network has achieved is less about assessing success or failure at any single point in time and more about continuous learning and adaptation in order to accelerate progress toward our goal.

Evaluate networks collaboratively. Engage network participants in developing a system wide picture of what is being tried and achieved by the various players. If the network builds a shared vision and theory of the change it would like to see, it becomes possible to collectively develop shared indicators that can be used to track progress.

Build capacity for ongoing learning and evaluation. Because networks are ever-changing and leadership, at its best, is distributed, participants across the network need to be constantly gathering feedback on what works and acting on it, individually and collectively. One way to do this is to invest in feedback loops and learning systems for ongoing assessment that help everyone build understanding together. This ensures real-time feedback, engages network participants in an ongoing strategic conversation and helps strengthen ownership of the network.

Learn openly and with others. For many grantmakers, there is little latitude for failed grants — investments that don’t achieve their stated outcomes. In the network context, this risk aversion is especially problematic because network participants may decide to take action that’s different from what a funder may have originally hoped for, and groups working through a model of loose network connections can take a long time to evolve and deliver tangible outcomes.

What are the characteristics of a healthy network?

Value. Effective networks offer multiple doors of entry — a range of value propositions that will resonate with diverse motives for participation. They also outline clearly for participants what can be expected from the network and what will be expected of participants in return.
Participation. Participants in healthy networks connect with others and engage in network activities. An environment of trust and reciprocity is nurtured through distributed leadership and an established code of conduct.
Form. A network’s form should reflect its purpose. For example, if its purpose is innovation there should be a large periphery — individuals loosely connected around the edges of the network that bring in fresh ideas.
Leadership. Leadership in healthy networks is shared and distributed widely. Ideally many participants are exercising leadership by weaving connections, bridging differences and inspiring others to recognize and work toward shared goals.
Connection. Connectivity throughout the network should be dense enough that the network will remain strong even if highly connected participants leave. Ample, well-designed space (for online and in-person contact) and effective use of social media can facilitate these connections.
Capacity to tap the network’s assets. Healthy networks operate on the premise that the assets they need are resident within the network or, if they are not, they have systems and habits in place for funding capacity — such as talent, resources and time — and bringing it in.
Feedback loops and adaptation. Networks are dynamic; what is needed and works today may not be relevant tomorrow. Healthy networks have feedback loops in place that enable continuous learning about what works and adapt based on their new knowledge.

Conclusion

Measurable network results can take a long time to materialize and may differ from a funder’s original intent when awarding a grant. Ongoing and learning oriented assessment can help the network evolve and adapt while also keeping the funder abreast of current and emerging social impact potential. Ultimately, networks take patience and as Roberto Cremonini, former chief knowledge and learning officer at the Barr Foundation shared, “…as networks grow, they build upon many small acts of relationship-building, problem solving and knowledge-sharing. Over time, these small acts build confidence within the network and position it for even greater potential.”