Oversight, Right, and More Mishegas

      Dear Dr. Steve: While I participate in my facility’s quality improvement efforts, I am puzzled about the approach. It seems to be almost exclusively based on survey issues and CMS quality measures. The staff and management seem to want to stick closely to the CMS script about how to interpret measures, and they seem reluctant to delve too deeply into performance and practice issues that the QAPI approach doesn’t cover. For example, they don’t want to look beyond the measure on antipsychotics to consider the many adverse consequences of all categories of medications. What should I do?
      Dr. Steve Responds:
      When I took over this column late last year, I noted the breadth and quality of the columns of my immediate predecessor, Dr. Jeffrey Nichols. This month’s column will call upon his October 2019 column (“Measurement, Meretricious Metrics, and Mishegas”) regarding quality measurement and improvement activities. We will then consider what everyone can do to amplify the limited and only modestly useful results of current Quality Assessment and Performance Improvement (QAPI) approaches.

      How Does the Survey Process Attempt to Assess Quality?

      Providers and practitioners have criticized the Omnibus Budget and Reconciliation Act of 1987 (OBRA) survey process on many accounts. One critique is that the survey process finds and cites many relatively minor things based on isolated process problems or episodic errors. Another critique is that it doesn’t really evaluate — and even minimizes — many aspects of care that provide value and improve quality of life.
      The Centers for Medicare & Medicaid Services has developed several approaches for assessing and trying to improve nursing home quality: (1) using quality measures based primarily on aggregated Minimum Data Set (MDS) data to summarize and compare performance on quality measures; (2) using the survey process to evaluate the care of individuals based on numerous quality indicators; and (3) using outside agencies and organizations (e.g., Quality Improvement Organizations [QIOs]) to advise and guide care and practices.

      Quality Measures

      Over the years, researchers and quality measurement specialists have developed measures and indicators for many aspects of health care. Allegedly, these reflect the extent to which facilities and practitioners give high-quality care.
      In the nursing home survey, these results are used extensively to select the survey sample and to guide surveyors on what to focus on in order to assess actual care. Surveyors receive a list of residents as triggers for various quality indicators and overall statistics about where the facility stands — for example, its antipsychotics utilization rate. They then apply this information to identify residents and issues to review during the survey. They can also add a limited number of cases identified during the survey.
      As Dr. Nichols noted, most QAPI approaches in post-acute and long-term care — including the CMS approach — have become heavily numerical (quantitative) and are based almost entirely on MDS data. He made the following salient points:
      • The use of computers to generate massive amounts of data has escalated without any comparable improvement in the quality of that data.
      • The proposed cure for meaningless and useless data seems to be more data.
      • There are so much data, and the utility of much of the data is so unclear, that it challenges everyone to try to understand what is meaningful or real.

      Guidance and Pathways

      Surveyors are required to perform specific survey tasks, including the SNF Beneficiary Protection Notification Review, and reviews of dining, infection control, kitchen, medication administration, medication storage, resident council meeting, sufficient and competent nurse staffing, personal funds, environment, and resident assessment (“Long Term Care Survey Process [LTCSP] Procedure Guide,” Feb. 6, 2021,
      Over the past few years, CMS has prepared a group of Critical Element Pathways (CEPs) to assist surveyors in reviewing specific aspects of care, including dining, nutrition, environment, infection prevention control, activities, activities of daily living (ADLs), behavioral and emotional status, abuse, and neglect. The full CEP list is available on the CMS website (part of a zip file entitled “Survey Resources with Staff Vaccine Documents (02-22-2022) (ZIP),”
      Each CEP guides surveyors with numerous questions related to specific processes and practices. For example, the nutrition CEP ( is used to evaluate individuals who “are not maintaining acceptable parameters of nutritional status” or who are “at risk for impaired nutrition.” It asks surveyors to (among many other things) “observe the resident at a minimum of two meals” and check for things such as hand washing, diet adherence, portions, assistance, environmental concerns, supplements, and whether the dignity of the resident is maintained.
      Surveyors are then guided to interview residents and families about how goals and plans were developed related to nutrition. They are even supposed to ask individuals with weight loss why they think they have lost weight (e.g., taste, nausea, dental, grief, or depression issues), whether they get enough time to eat, whether they have been asked about their food and mealtime preferences, whether they get the right diets, and whether they get assistance with eating or encouragement as needed.
      Surveyors are instructed to interview nursing aides, dietary aides, paid feeding assistants, and the dietitian or dietary manager. Finally, they are guided to interview relevant medical practitioners but only “if the interventions defined, or the care provided, appear to be inconsistent with current standards of practice, orders, or care plan.”

      Drawing Conclusions

      As the survey proceeds, surveyors are supposed to use the information they have gathered through interviews, observations, and record reviews to draw conclusions about compliance. That is, did the facility meets the requirements?
      On paper, this appears to be very comprehensive and likely to identify deficient practices, support any survey deficiencies, and accurately reflect facility care quality. But, as former U.S. Attorney General Benjamin Brewster allegedly stated, “In theory, there is no difference between practice and theory, while in practice there is.” The question remains: How do they interpret the information they collect?

      Measurable and Meaningful

      The survey focuses on whether facilities qualify to receive Medicare and Medicaid payments by complying with related regulations. For example, if the surveyor finds several process variations in the care of an individual who lost weight (e.g., a delay in dietitian notification and a noisy dining area), they will likely cite those as deficiencies without considering the overall picture, or knowing that medications commonly cause anorexia, or acknowledging that appetite stimulants are not an acceptable approach in most cases.
      Despite having hundreds of questions in the CEPs and lots of data in the quality measures, surveyors have limited effective guidance for specific situations. Because they may not understand the impact of clinical decision-making processes on results, they often speculate about causes. Surveyors only occasionally delve into whether the care of individual patients is consistent with clinical standards of practice related to proper cause identification/diagnosis, treatment selection, and monitoring.
      Medical practitioners and medical care are not the purview of the survey. So while physician performance (e.g., responding to calls or visiting residents) may be scrutinized, there is minimal scrutiny of whether they did the right thing correctly (e.g., identified and addressed medication-related adverse consequences and established correct diagnoses). The facility will get the citation for the poor result; the surveyors will typically focus on the usual findings (e.g., staffing, approach to residents, or procedural errors) to make their decisions.
      Dr. Nichols noted that the result of too much data and too little understanding “has become a system-driven collection of information intended to measure quality, productivity, and individual variance from system standards (bad) or compliance with system expectations (good).” For example, Dr. Nichols discusses the questionable significance and impact of antipsychotics quality measures. The limited “all-or-none” approach (either on one or not) does not reflect clinical reality or provide a meaningful picture of a facility’s care of behavior and psychiatric issues. Instead, it promotes a distorted and robotic approach to care that interferes with achieving desired clinical outcomes as opposed to artificially created ones (e.g., fewer antipsychotics somehow implies better care quality). The result is “mishegas” (foolishness).

      What Can We Do?

      Currently, there is little quality oversight beyond quality measures, the survey process, very general QIO advice, and the latest CMS initiatives. Because few if any current common quality measurement and oversight agents perform meaningful reviews of most clinical practices in individual patients, it is unlikely that anyone will ever identify whether care in our facilities is consistent with applicable clinical standards of practice unless we do it ourselves (see my 2014 editorial “Survey Deficiencies Are Not Reliable for Drawing Conclusions About Aspects of Quality,” J Am Med Dir Assoc 2014;15:82–84).
      Surveyors often miss important things and may reach questionable conclusions. But then, so do facilities and their staff, management, and medical directors.
      Ultimately, facilities that rely too heavily on the currently mainstream QAPI approaches are unlikely to find and correct any issues in the underlying care process, clinical reasoning, and problem-solving that profoundly influence outcomes. For example, the current approaches are inadequate to help you to improve a facility’s medication utilization.
      Facilities and practitioners can incorporate CEP questions and probes into their quality oversight activities. However, they must go well beyond the survey process and methods to interpret the data, pinpoint causes, and respond appropriately to the findings. This can be done effectively and efficiently by simultaneously discussing individual patients while also probing the underlying care processes and clinical decision-making. I have developed a process known as “Smart Case Review” that can help with this (J Am Med Dir Assoc 2021;22:2212–2215; see also: Caring for the Ages 2022;23(2):18).
      Do not jump quickly to process-related conclusions without giving ample consideration to problems of clinical practice — such as not identifying or addressing medication-related adverse consequences or medical conditions that affect appetite and weight, or following inappropriate dietitian recommendations to initiate appetite stimulants.
      For example, concerns have been published over time regarding management of dysphagia, including the excessive use of modified diet texture and fluids and unnecessary restrictions (J Am Med Dir Assoc 2008;9:292–301; J Am Med Dir Assoc 2019;20:952–955). Altered diet and fluid consistency has limited efficacy, and there is often substantial harm resulting from excessive imposition of diet modification in individuals who do not want or need it. In my own experience, surveyors essentially never cite facilities for such excesses, including flagrant rights violations related to imposing unwarranted and undesired diet and fluid texture restrictions. The surveyors regularly overlook actual evidence-based practice, despite their inclusion in the State Operations Manual (e.g., see the discussion of modification of food and fluid consistency under “Functional Factors” on p. 328,
      QAPI meetings should not just run through numbers or treat numbers and topics in isolation. Analyze and address issues (staffing, weight loss, falls, behavior) jointly because multiple issues often have common causes and there are often multiple causes of individual issues. And finally, don’t ever ask surveyors to advise on how to improve care and correct issues; they are not authorized or able to identify all relevant issues or distinguish causes.
      To reprise Dr. Nichols’ thoughts, some of what is meaningful is being measured, but much of it is being skipped. And while some of what is currently being measured may be clinically significant, current approaches to identifying and addressing related causes are far from optimal. It would not be hard to do better, but we are going to have to lead the way. If we do not seek and address these process and practice issues, no one else is likely to do so.
      Dr. Levenson has spent 42 years working as a PALTC physician and medical director in 22 Maryland nursing homes and in helping guide patient care in facilities throughout the country. He has helped lead the drive for improved medical direction and nursing home care nationwide as author of major references in the field and through his work in the educational, quality, and regulatory realms.