The National Strategy for Preventing Veteran Suicide 2018 – 2028 identifies suicide as a national public health issue. VA strives to provide high-quality health services to Veterans throughout their lives, including mental health services through numerous mental health and suicide prevention programs at VA medical centers and clinics. Nine of these programs are selected each year for evaluation and fall within distinct categories of mental health care, substance use disorder treatment, opioid prescription guidelines, services for women Veterans and suicide prevention. Findings from these evaluations are compiled in a comprehensive annual report to Congress.
The Clay Hunt Act’s objective is to evaluate mental health care and suicide prevention programs to identify what works best. Specific measures of effectiveness for each program continue to be assessed and defined as needed, and subsequent analyses built on prior year evaluations. VA is also required to assess how well VA providers follow the VA Clinical Practice Guidelines (CPG) for Opioid Therapy for chronic pain. VA partnered with Aptive to conduct all these third-party evaluations.
Aptive has extensive experience with evaluation methodology, data analysis and project management. Under the Clay Hunt Act contract, Aptive conducts the nation’s largest suicide prevention and mental health program evaluation for the largest integrated health care system in the country—the Veterans Health Administration.
For the congressional reports produced so far, the first step was to thoroughly understand VA’s suicide prevention and mental health programs. To this end, Aptive reviewed VA’s literature on each program, including mental health handbooks and suicide prevention guides, and met with VA subject matter experts. These experts provided insight about program goals, effectiveness, patient access to care, relevant databases or metrics that could be used in the evaluation, how the COVID-19 pandemic potentially impacted programs and other pertinent program nuances and information. Through these interviews and information gathering, we developed strong awareness of the mental health care and suicide prevention mission and vision of VA experts and clinicians.
Designing and executing a comprehensive evaluation methodology and analysis plan
The evaluation team develops a comprehensive evaluation methodology and analysis plan; research questions about clinical outcomes; Veteran satisfaction and cost-effectiveness; and metrics, data sources and analytic methods.
We evaluate VA mental health care delivery in both inpatient and outpatient settings over the years to reduce suicide ideation and prevent mortality, as well as study the impact of social stigma and other factors on receiving care.
Aptive acquires the requisite health data for analysis and tests each data set for integrity and validity before conducting analyses. To analyze program effectiveness, we also measure target conditions, outcomes, intensity of care, comorbidities and other factors. To meet Clay Hunt Act requirements, we also independently evaluate VA health care system opioid prescribing practices compared with guidelines and recommendations set by other federal agencies. Our analysis differs from year to year in complexity and evaluation design.
A critical component of the evaluation process is recommending best practices based on what we’ve learned. Therefore, we consulted more than two dozen mental health experts both within and outside VA. With this invaluable input, we drafted best practices recommendations and validated them through a panel of subject matter experts. Once Aptive and the panel finalized the recommendations, we compiled the information into the Annual Report to Congress.
Aptive completed complex data analyses built on prior year evaluations. This year’s analysis focused on post-COVID impacts on mental health programs. Aptive experts continue to collect, process and prepare data for accurate analysis.
After following the evidence-based methodology and consulting with subject matter experts, Aptive created and submitted:
At more than 400 pages, the Annual Report to Congress details the entire evaluation process, including recommendations for continuously improving mental health services delivery for Veterans. Future analyses will build on the strong working relationships between VA and Aptive and incorporate additional metrics to produce an even more comprehensive study of VA’s mental health and suicide prevention program effectiveness.
