Why We Measure: Inside Ruby Hill Productions' Entertainment-Education Evaluation Framework

Most community arts organizations count heads and call it impact. Ruby Hill Productions built something different: a dual-track evaluation architecture grounded in fifty years of behavioral science — rigorous enough for academic publication, practical enough for a founder-led organization in Year One. This post is an honest account of what that framework is, why it matters, and what it means for the communities we serve.

The problem with counting seats

Every grant report in the performing arts eventually asks: how many people attended? It is the easiest number to produce and, by itself, the least meaningful. An audience of 300 people at a play about immigration policy is not, on its own, evidence of anything except that 300 people were in the room. What changed in them? What did they do afterward? What moved in the community as a result?

The most common evaluation failure in community arts is to report outputs — performances delivered, audience members served — and call them impacts. Ruby Hill Productions' evaluation framework is built on a foundational refusal to make that mistake.

We distinguish rigorously between three tiers of evidence. Outputs are what we did: performances, workshops, teaching artists employed, community events produced. Outcomes are shorter-term changes: shifts in individual knowledge, attitudes, self-efficacy, and behavioral intention — measurable within six months of a production. Impacts are longer-term, broader transformations: changes in community discourse and norms, civic action taken by audience members, coalition relationships strengthened, policy arenas influenced. The distinction is not semantic. It determines what we measure, how we report it, and which funders find our evidence credible.

Two tracks, running simultaneously

RHP evaluates every production at two levels: the individual level and the collective level. We call these Track A and Track B.

Track A captures Entertainment-Education (EE) effects on individual audience members. The instruments we use — the Transportation Scale Short Form, the Narrative Engagement Scale, Cohen's Character Identification Scale, the EORM Resistance Battery — are validated psychometric tools drawn from fifty years of behavioral science research. They tell us: Was the audience absorbed in the story? Did they identify with the transitional character? Did the narrative lower their psychological defenses against the message? Did their self-efficacy — their belief in their own capacity to take a specific action — shift? Track A is the domain of Sabido, Bandura, Singhal, Green, and Moyer-Gusé. It speaks to health funders in the language they trust: behavior change, attitude change, pre/post measurement, validated scales, Cronbach's alpha.

Track B captures media impact at the collective level. Borrowing Philip Napoli's 2014 synthesis of the Norman Lear Center's Media Impact Project, we track what happens beyond the individual viewer — in the community, the civic landscape, the coalition of organizations working on related issues, the public discourse. Track B asks: Did the production generate media coverage that rippled outward? Did advocacy organizations engage with our work and use it as a tool for their own efforts? Did audience members take civic action — donate, organize, contact a public official — in the weeks after leaving the theatre? Track B speaks to community development funders, arts funders, and academic partners in the language they trust: coalition effects, narrative change, civic engagement, contribution analysis.

Neither track alone is sufficient. A production might generate measurable attitude change in every audience member (Track A success) but fail to create any durable community-level shift (Track B flat). Or a production might galvanize a coalition of organizations and generate significant media impact (Track B success) without producing measurable individual effects (Track A results unclear). Running both tracks simultaneously gives RHP — and our funders, and our community — a complete picture.

The science of why stories change minds

The behavioral science underlying our framework is worth understanding, not just as a credential but as a genuine argument for why theatre is the right instrument for the work we do.

Miguel Sabido, the Mexican television producer who created the modern Entertainment-Education (EE) methodology at Televisa in the 1970s, discovered something counterintuitive: stories change behavior more reliably than information. He based his methodology on three character archetypes — positive role models, negative role models, and transitional characters who evolve across the narrative arc — and on the epilogue, which links the drama to real-world resources. The evaluation implication: we design our productions with these archetypes deliberately, and we measure which archetype audiences identify with.

Albert Bandura's Social Cognitive Theory explains why. Behavior is learned through observational learning of models, mediated by self-efficacy (the belief "I can do this") and outcome expectations (the anticipated consequences of acting). When an audience member identifies strongly with a transitional character who faces a challenge similar to their own and navigates it successfully, their self-efficacy for that behavior increases. They have, in effect, practiced the behavior in their imagination. This is not metaphor — it is the psychological mechanism that makes EE effective. We measure it directly: pre/post self-efficacy items, custom-built for each production's focal behavior.

Melanie Green's Narrative Transportation Theory adds a crucial refinement: when audiences are cognitively, emotionally, and imagistically absorbed in a story — when they are "transported" — their real-world beliefs shift toward the story's implied beliefs. Critically, this effect is larger for high-quality productions. Artistic craft is not decoration; it is the mechanism of impact. A better-written play, a more truthful performance, a more precise dramaturgical choice — these translate directly into higher transportation scores and, therefore, stronger attitude change. This is the scientific argument for our commitment to professional standards in community theatre. We do not use "community" as an excuse for mediocrity.

Emily Moyer-Gusé's Entertainment Overcoming Resistance Model (EORM) explains why entertainment reaches people that direct messaging cannot. Audiences resist direct messages through three mechanisms: reactance (being told what to think triggers psychological rebellion), counterarguing (audiences dispute the message as it is delivered), and perceived invulnerability ("this issue doesn't apply to me"). Narrative entertainment defeats all three simultaneously through transportation. Our post-show survey includes a resistance battery measuring all three subscales. Low resistance scores combined with high identification and transportation scores are strong evidence of EE effectiveness — that the production did what EE theory predicts it should.

The engagement hierarchy: from passive to action

One of the most practically powerful frameworks in our evaluation architecture is the Harmony Institute's engagement hierarchy, synthesized by Napoli (2014) and adapted by RHP for live theatre. It distinguishes four levels of audience engagement:

Passive: attendance, website visits, social media follows.

Expression/Sharing: post-show verbal comments, social media shares, user reviews.

Participation: talkback participation, program sign-ups, inviting others, volunteering.

Action: donating, civic organizing, contacting a public official, joining an advocacy effort.

We track the ratio of Action-tier to Attendance-tier engagement for every production. A show that draws 200 audience members and generates 40 documented Action-tier engagements is performing at 20% — meaningfully different from a show that draws 200 and generates 4 (2%). This ratio is a key performance indicator we report to every funder, because it captures not just who was in the room but what they did with the experience afterward.

The community advisory council IS the evaluation

One of the principles most central to our evaluation framework — and most often absent from community arts evaluation generally — is the requirement that the communities most affected by our programs have the most voice in shaping how those programs are evaluated. "Nothing about us without us" is not only a disability rights principle. It is an evaluation ethics principle.

RHP's Community Advisory Council (CAC) plays an active role in evaluation design, not just program review. Before any production, the CAC reviews proposed survey instruments and flags items that may be culturally inappropriate, linguistically inaccessible, or potentially harmful to specific communities — particularly undocumented immigrants, LGBTQ+ youth, and disabled community members. The CAC participates in participatory indicator development: the community helps define what "success" looks like before RHP designs the instrument that measures it. CAC members are compensated for this work, because asking communities to labor for free in the service of organizational accountability is itself a form of extraction.

This approach is grounded in the CARE Principles for Indigenous Data Governance (Carroll et al., 2020) — Collective benefit, Authority to control, Responsibility, Ethics — and in the broader principle, following Wallerstein & Duran (2010), that evaluation is not something done to a community but something done with and for it. Communities that have been extracted from, surveilled, and discarded by institutions are right to be skeptical of evaluation processes. RHP earns the right to measure by demonstrating that measurement serves the community, not just the organization's grant reports.

The "no data without action" commitment

RHP makes a standing commitment to every evaluation participant and every community member involved in evaluation design: we will not collect data unless we are prepared to do three things.

First: share results back with participants and the community before reporting to funders or publishing findings. Second: take at least one documented organizational action in response to evaluation findings per production cycle. Third: tell participants what changed as a result of their input — in plain language, in accessible formats, and through the channels the community actually uses.

This is not a policy. It is a cultural commitment. We believe evaluation should be a form of listening, not surveillance. Every survey we distribute, every talkback we document, every coalition effect we track — these are acts of attention paid to people who have historically been studied without being served. Our obligation is to close that loop.

What this means for communities in Southwest Denver

The practical implications of this framework are worth naming directly. For audiences at Ruby Hill Productions shows, it means: you will be asked, briefly and anonymously, about your experience — not to satisfy a grant requirement, but because your experience is the evidence base for our claim that theatre changes lives. The survey takes fifteen minutes. Your answers shape how we design the next production.

For community organizations working on immigration, LGBTQ+ rights, disability justice, racial healing, or youth development — the issues our productions will address — it means: RHP will come to you before we open, not after. We want to know who your constituents are, what they need, what a production touching on your issue could do for your advocacy work. We will track what your organization does with our production. We will report that in our impact narrative. We want to be a tool in your toolbox, not a parallel enterprise.

For funders: we can speak your language. Health funders will receive pre/post SCT measures, EORM resistance results, and validated psychometric scales with reported Cronbach's alphas. Arts funders will receive WolfBrown Intrinsic Impact constructs, engagement hierarchy data, and the Action-to-Attendance ratio. Community development funders will receive coalition effects documentation, Most Significant Change stories, and Ripple Effects Mapping outputs — centering community member voices, not organizational self-promotion. National and academic partners will receive the full dual-track methodology, theory of change documentation, and a contribution analysis narrative.

We will not always have clean answers. We are a Year One organization. Our sample sizes will be small. Our Track B documentation will sometimes catch more smoke than fire. We know that. The framework exists not because we already have all the evidence, but because we are committed to building it rigorously and honestly — and to sharing what we find, including the findings that complicate our own claims.

Conclusion

There is a version of community theatre that treats evaluation as a bureaucratic obligation — a set of boxes to check before submitting the next grant report. Ruby Hill Productions is not building that version. We are building an organization in which evaluation is structural: baked into production design, led by community, grounded in validated science, and oriented toward the people we serve rather than the funders we report to.

The science says that stories transported audiences change minds. That characters who look like you on stage increase your belief in your own capacity to act. That theatre, at its best, does not just move people emotionally — it moves them behaviorally and civically. We believe that. We intend to prove it.

This is why we measure.

—

Ruby Hill Productions' Entertainment-Education Evaluation Manual, Version 2.0 (2026), is the source document for the frameworks described in this post. All instruments referenced — TS-SF, NES, CIS, EORM, self-efficacy scales, social norms measures, and the full suite of community health and wellbeing instruments — are catalogued with full item text, reliability data, licensing information, and plain-language summaries in that manual.