How Do You Measure an Experience?

from Director of Quality Assurance Lynn Pilcher

People with torches gather in the dark woods to sing songs and celebrate Yule

How do you measure the experience of a stay at Spring Lake Ranch? Some things simply defy measurement. Take for example, the recent Spring Lake Ranch annual tradition of Yule, in which the community gathers to celebrate the return of the light on the darkest night of the year. With each sharing of song, poetry, and personal self-expression a candle is lit, the powerful symbolism that each gift shared and received, helps to create the light of healing, hope, acceptance, and belonging.

To measure the impact of this magical moment is difficult indeed, but it is precisely this kind of experience which we believe sets Spring Lake Ranch apart from other residential mental health treatment and dual diagnosis programs. The bedrock of the SLR program is formed on the belief that each person has something to contribute, and all are accepted. Our core values of hope, personal relationships as the currency of healing, and a connection to the land infuse the unique therapeutic approaches of work, community, and professional expertise. But does this approach work, and if so, can we measure it?

The therapeutic community model which SLR closely resembles has been around since World War II, and yet the efficacy of the model has only minimally been studied. There are many wonderful qualitative studies, such as this 2008 article written by Barbara Dickey, PhD and Norma Ware, PhD, focused in part on Spring Lake Ranch; however, there is a need for more understanding on the effectiveness and lasting transformative impact of the therapeutic community model of care.

Beginning in July 2022, Spring Lake Ranch initiated an Outcomes Advisory Group to begin to ask and answer some of these questions. Under the umbrella of Quality Assurance, which looks at program improvement and resident experience, the group’s first task was to look at two years of data using an assessment tool called the DLA-20. Essentially, this tool looks at 20 areas of daily activities used in everyday life, but ones which often get interrupted or never developed due to mental illness and substance use. Assessing a resident’s competence in these areas on admission and then again at discharge forms the basis of good treatment planning. It also informs program evaluation in which we can learn where the program has the most influence and areas where the program can increase its efforts.

A report, written by former House Advisor Kim Van Orden, PhD, draws upon data collected between 2019-2021. The report explores three areas: 1) the degree of improvement in functioning demonstrated by residents at SLR from admission to discharge; 2) domains of functioning most responsive to SLR programming; and 3) potential areas for future program evaluation and enhanced quality improvement strategies.

Graph of 20 Domains of Functioning

For me, highlights from this report include the following:

  1. The benchmark set by the developers of the assessment indicates that a standard to seek for global improvement of functioning should be at least 35%. The Ranch’s data for improvement in global functioning was over double that, with 76% of the residents demonstrating clinically significant improvement in global functioning.
  2. The area with the lowest average score upon admission was in productivity. While it wasn’t the highest score overall upon discharge from SLR, productivity was among the highest rate of change of all 20 areas of daily activity for our departing residents. This underscores that our therapeutic approach engages people who report having challenges around work in experiences of productivity.
  3. The data showed the areas with the greatest magnitude of improvement were self-management of health conditions, communication skills, time management, productivity, money management, problem solving, and leisure skills. As noted in the report, these “are domains of functioning that are directly targeted by the therapeutic programming at SLR, suggesting that improvement is likely due to participation in the SLR program, rather than passage of time. In particular, the structure of the work program and scheduled leisure and recreational activities help residents regulate their social rhythms, provides opportunities for behavioral activation, and allows for productive engagement in a community that becomes routine.”

I encourage you to read the report and let us know what you think. This report forms the first of an ongoing, annual look at our outcomes. In 2023 we will be adding some self-report measures to complement the DLA-20 data, as well as a post-discharge survey. For more information or to see how you can be involved, please let me know. I’d like to take this opportunity to thank the Outcome Advisory Committee members, Jennifer FauntLeRoy, MD; Monica Lawson, PhD; Kim Van Orden, PhD; Kevin Molloy, and Vanessa Granados.

CARF Accredited: Spring Lake Ranch programs are CARF accredited. The CARF accreditation signals our commitment to continually improving services, encouraging feedback, and serving the community.

Spring Lake Ranch is a member of the American Residential Treatment Association (ARTA). ARTA members are dedicated to providing extraordinary care to adults with mental illness.