4 parameters for evaluating your community engagement efforts

Introduction

In 2022, one of Zencity’s partner communities in Virginia considered constructing a roundabout as a traffic calming measure. Before making the decision, the city organized a town hall meeting to present the plan to residents and get their feedback. The room was packed, with residents lining the walls, clamoring for the microphone, and raising their voices in an attempt to be heard. Clearly, there was no shortage of feedback on the roundabout proposal. The City Manager and Council members were delighted at the turnout and the level of engagement; who knew residents are so passionate about roundabouts? (Just kidding; we knew).

When it came time to go over the meeting notes and analyze the feedback provided, the city realized that they had little insight into where the community actually stood on this issue; were they more in favor or against the roundabout? This was puzzling, given the participation numbers. How was it possible that with so many people attending and speaking up, there was so little actionable data? 

A packed room, hundreds of comments on a city post, thousands of survey responses, a city inbox crammed with resident emails; these appear like hallmarks of successful community engagement efforts, glistening with the promise of rich resident feedback.

However, participation is only one, limited, parameter of effective and impactful engagement. Sure, the community showed up, but participation levels alone fail to capture the quality of the feedback provided. 

What then, aside from sheer participation numbers, can help us evaluate the effectiveness and quality of your community engagement efforts? 

Here are 4 parameters you should consider when evaluating a community engagement action: Depth, Audience, Representativeness, and Effort – or DARE. 

Depth

What can the feedback teach you and what it can’t

The first parameter refers to the depth of input or level of detail the local government can get out of the data gathered in the engagement process. Feedback comes in many forms; a “like,” a “retweet,” a meme or a GIF, a long, well-reasoned comment detailing reasons for support or objection, a 1-5 scale score on a survey question. Clearly, each one of these provides a different level of insight.

A successful and effective community engagement effort will generate the depth of feedback that is most useful and relevant to you in this particular case: will an average score calculated based on basic-binary data be sufficient, or do you require rich discourse that is generated in social media discussions or in-person meetings to reach a decision?

Remember the town hall meeting concerning the roundabout? The city wanted to get a sense of support versus opposition but ended up with long tirades about roundabouts, and general arguments about road safety, traffic, and of course, potholes. This was not invaluable, but it was not the depth of data the city was after. In fact, in this specific case, a survey or a poll sent out to residents would have provided the desired level of data the city needed to make its decision. 

Therefore, ask yourself, “is the data I am likely to get through this engagement effort helpful and appropriately nuanced to support decision-making?” If you need a bottom line, an overall score outlining support or satisfaction, yet you have pages upon pages of resident commentary about their perceptions, preferences, and concerns, then this was not an effective community engagement – and vice versa.

Audience

Who are the crucial stakeholders, whose feedback is essential

In one Texas partner community, the city wanted to understand the resident sentiment around the possible recall of one council member. The city posted on its social media and encouraged residents to voice their opinions on the matter, but the leading comment was that this was not an issue for all residents to decide on, but for the residents of the council member’s district. “Are you making sure you’re listening to the feedback coming from this specific community,” commenters asked the city. “That is your audience.”

The second parameter examines the audience that participates in the engagement process. It can be a large or a small group, with a mix or a homogeneous representation of stakeholders. However, the crucial factor here is that the audience targeted and included in the engagement effort is relevant to the issue at hand. If this is an issue pertaining to a specific area in the city, the audience has to include residents from that area For example, if it is an issue affecting communities of color, they are a critical stakeholder and the engagement must be primarily targeted towards them, even at the expense of participation levels.

When you set out to evaluate this parameter, ask yourself whether the audience accurately and fairly reflected the most relevant stakeholders and if the feedback came from the right people.

Representativeness

Whose voices are heard and whose voices are marginalized

Representativeness refers to the question of whether the data shared in the engagement process is representative of the community, either statistically and/or in terms of equity and inclusion. This is a critical question irrespective of the issue at hand, as within every target audience you are likely to have a diverse range of community members, with different perceptions, opinions, and concerns.

An effective and successful community engagement effort will be designed to ensure that the feedback collected reflects a diverse range of voices, either within the general population or a specific group of residents. 

Ask yourself: “Is the feedback I collected accurately representative of my community and am I giving an equal voice to all community members?” There are several ways to achieve representativeness: A statistically-representative survey, engagement methods in different languages, and distribution and promotion across multiple channels. It is critical to create the necessary infrastructure to ensure that people of all age groups, backgrounds, socio-economic circumstances, racial and ethnic groups, gender groups, and faiths are able to participate and provide input.

Effort

How difficult (or easy) is it for residents to provide feedback

This is the easiest parameter to evaluate; it measures the level of effort participants had to invest to voice their opinion and provide their feedback. Did they have to find a babysitter, get in their car, drive to the town hall, and wait in line to say their piece? Or did they share their feedback by phone while walking their dog? Obviously, lower effort usually means fewer hurdles and therefore results in more feedback from a more diverse set of voices. This includes people who have less flexible schedules or can’t physically come to the meeting, and whose voices, as a result, are heard less often or become discounted.

By reducing the amount of effort required by the participant, the engagement process is more accessible and makes participation easier for harder-to-reach, marginalized, or vulnerable communities. 

Overall, the aim is to lower barriers to participation and make the process as effortless as possible. How? Make sure you offer multiple ways to provide feedback that suits diverse levels of lifestyles, preferences, resources, and capabilities.

Conclusion

By now, you may have realized that community engagement efforts that meet all 4 parameters are difficult to create. Representativeness can be best achieved in a statistically-representative survey, but it might mean compromising on the depth of the feedback. Ensuring that you have the appropriate audience might require more effort from participants.

Do not despair. The most effective community engagement effort is one that is designed with the four parameters in mind in a way that answers your specific objectives but is not necessarily perfect on all four. A variety of community engagement solutions is the best guarantee for a flexible and adaptable engagement strategy that can emphasize each parameter when it is most needed.

And herein lies the twist. These four parameters serve a dual purpose. Yes, they are a helpful prism through which to evaluate the engagement after the fact, but, more importantly, they are a crucial framework with which to design community engagement activities before they are implemented.

This 4 parameter approach to designing and evaluating enables local government organizations to identify areas for improvement and design more effective and inclusive engagement processes that reach a wider audience and gather diverse, and actionable, insights from their constituents.

Here are the questions you should DARE to ask yourself when designing your community engagement effort:

D - What is the Depth of input I require?
What type of data will be most useful to me moving forward – a bottom-line score or a nuanced overview of resident opinions and concerns?

A - Who is the most relevant Audience?
Given the issue at hand, who are the relevant stakeholders whose feedback is absolutely imperative and how do I make sure they are included?

R - How do I ensure and encourage Representativeness?
What can I do to ensure that my engagement is inclusive and equitable? What are the tools and methodologies that would contribute to a diverse and inclusive session?

E - How much Effort am I asking of participants?
Given the audience, what is the level of effort I can and should expect? How do I make sure that there are adequate and accessible methods for sharing feedback?

The names that appear in images and social accounts are fabricated for the purpose of an example in this guide.

Learn how you can implement this strategy in your city