The UNOS Evaluation and Quality Department monitors member compliance with OPTN policies. Our goal is to ensure the greatest possible equitable treatment of all patients. To date, the national rate of compliance with OPTN policies, by member institutions is extraordinarily high.

To ensure continuation of these high rates of compliance and and to promote further improvements by member institutions, the Evaluation and Quality Department manages:

• review of patient listings, administration of the RRB, and evaluation of member data submission
• organ allocation review
• on-site review of OPTN members

Evaluation and Quality continues to develop and refine our processes to assist member institutions and promote patient equity. If you have specific questions regarding policy compliance, please contact UNOS at (804) 782-4800 or e-mail us.

Review of Patient Listings, Administration of the RRB, and Evaluation of Member Data Submission

Review of Patient Listings and Administration of the RRB

When a center lists a liver, heart, or intestine candidate for a transplant, it must submit a justification form. This form documents the severity of the patient's illness and justifies the status at which the patient is listed. Justification forms are currently submitted in electronic versions.²

Urgent status listings for liver and heart patients are forwarded to the appropriate Regional Review Board (RRB) for consideration. Liver RRB's review listings for all liver candidates in Status 1, special case exceptions for MELD/PELD liver candidates, and hepatocellular carcinoma (HCC) candidates. Thoracic RRB's review listings for heart candidates in Status 1A and special case heart candidates in Status 1B. These boards also consider appeals of cases initially turned down for a particular medical urgency status.

RRB reviews and appeals for heart Status 1A and 1B are conducted via facsimile. RRB reviews and appeals for liver Status 1 and MELD/PELD are conducted electronically through the UNet^sm  system.

With the implementation of MELD/PELD in February 2002, the RRB began reviewing cases requesting additional MELD/PELD points for special case exceptions. The RRB retroactively reviews all cases requesting additional MELD/PELD points for cases involving hepatocellular carcinoma (HCC).  For cases involving hepatopulmonary syndrome, familial amyloidosis, and primary oxaluria, RRB review is prospective. Pediatric patients with metabolic diseases (i.e., Crigler-Najjar Disease Type I or OTC) or non-metastatic hepatoblastoma require retrospective review for Status 1 and prospective review for requests of additional PELD points.

For all prospective reviews, the RRB has 21 days after a center has requested additional MELD/PELD points for this type of patient to determine a majority vote. If a majority vote decision has not been determined within 21 days, the patient's MELD/PELD score defaults to the original score requested by the center. These cases are referred to the OPTN Liver and Intestinal Organ Transplantation and Membership and Professional Standards Committees (MPSC) for further review.

Beginning September 2002 all required data fields on the justification form must be submitted online through UNet^sm before patients can be added, upgraded or extended. Moreover, under this system patients will automatically be downgraded unless they are re-listed by an attending physician within the time frames described in the definitions of current UNOS policy. Evaluation and Quality staff will be calling each center prior to a patient's expected date of justification form submission to remind the center that the form is due and that the patient will be downgraded unless the form is submitted.

Evaluation and Quality auditors verify, during on-site reviews,  Status 1A heart listings, Status 1 liver listings, MELD/PELD HCC and special case exceptions, MELD/PELD scores greater than or equal to 20, and the MELD/PELD score at the time a transplant offer is made.

Evaluation of Member Data Submission

Pursuant to new OPTN policy, each transplant center, OPO, and histocompatibility laboratory must complete and submit 95% of expected forms within three months of the due date, and 100% of expected forms within one year of the due date. These standards apply to all data collected on forms and donor referral information submitted to the OPTN. One hundred percent of the potential transplant recipient (PTR) refusal code data must be submitted within 30 days of a match run date. 

In order to facilitate the implementation of the new data submission standards, the Board of Directors approved an Amnesty Plan at its June 2002 meeting. Under the Amnesty Plan, all Transplant Recipient Follow-up (TRF) Forms, except for the one-year, three-year, death/graft failure, or most recently expected follow-up form, generated before June 30, 2002³  were granted a "forgiven" status in UNet^sm meaning that the submission of these forms would not be monitored for compliance. All other forms, generated before June 30, 2002, are given a one-year deadline of June 30, 2003 for submission.

Pursuant to the Amnesty Plan, the Evaluation and Quality Department is monitoring the OPTN members with outstanding forms for submission of forms by the June 30, 2003 deadline.  All members with outstanding forms have received status letters with a detailed list outlining the current outstanding forms. The Evaluation and Quality Department will send letters after six and nine months informing outstanding members of their status. After the June 30, 2003 deadline, members with outstanding forms will undergo an internal due process with the Director of the Evaluation and Quality Department with potential referral to the MPSC for review.

Outstanding Forms Reports are still prepared monthly for primary data coordinators and bi-monthly for program administrators and directors, histocompatibility lab directors and OPO executive directors. The report provides summary information on the number of forms overdue by 1-3 months, 4-6 months, 7-9 months, 10-12 months, and greater than 12 months. The report also includes information on compliance with data submission policies so that members receive information on the specific forms that are outstanding for their facility, as well as an indication of how they are meeting the policy standards.

In addition to the outstanding forms report, UNOS staff contacts members when their forms are overdue by at least 3 months. Institutions with forms that are more than 6 months overdue are referred to the Evaulation and Quality Department for further action. The data subcommittee of the MPSC reviews PTR forms submission data and may determine that a letter be sent to the OPOs that have not submitted 100% of their expected PTR forms and have failed to respond to requests from UNOS staff.

Organ Allocation Review

The Evaluation and Quality Department monitors the organ allocation process to ensure that policies matching organs with waiting patients are followed. This monitoring includes:

• an average of 1,300 cases per month and an additional 300 mandatory shares per month
• verifying offers made pursuant to the match list
• noting changes in donor information between match runs
• producing a zero-antigen mismatch/payback report on a monthly basis
• reviewing refusal codes on match runs
• revising practices as policies are amended or adopted
• special studies as requested by the OPTN committees

Evaluation and Quality works closely with other departments, such as Clinical Data Systems and the Organ Center to ensure that possible allocation policy violations are identified. After researching all potential violations, the department presents blinded case histories to the Policy Compliance Subcommittee of the MPSC for further consideration.

The UNOS Evaluation adn Quality Department also monitors any trends or developing issues in allocation policy across regions and among members. Any observed issues or problems are summarized, analyzed, and forwarded for review to the applicable OPTN committee for clarification, interpretation, or a policy change.

On-site Reviews of OPTN Members

On-site reviews are a key component in evaluating a member's compliance with OPTN policies. Through the on-site review, staff auditors assess how well transplant center programs and OPOs adhere to listing and allocation policy. UNOS staff offer recommendations for improvement when indicated.

Between October 2000 and September 2003, UNOS staff will conduct on-site reviews of every heart and liver transplant program in the U.S. The Evaluation and Quality Department has reviewed 203 programs at 132 centers since October 2000¹ and will have reviewed approximately 213 programs at 140 centers by the end of 2002. The auditors will have reviewed the medical records of approximately 2,008 heart patients and 2,394 liver patients.

On-site reviews of other organ transplant programs for compliance with patient notification requirements are being conducted in conjunction with review of a same center's liver and heart programs.   

Evaluation and Quality staff began review of OPOs in February 2002 and has conducted 10 OPO on-site reviews.  Evaluation and Quality will review approximately 15 OPOs on-site and 175 donor records by the end of 2002.   

Selecting Institutions for On-site Reviews

Selecting institutions and setting the visit schedule is based on the following factors:

• institutions with previous violations for which follow-up visits are requested by the MPSC
• centers that have had two or more patient status listings turned down by the regional review board
• liver programs with extensions greater than 14 days for patients in Status 1
• evidence of possible policy violations that have not been resolved satisfactorily
• institutions that have not been visited within the last three years
• centers for which more than 5% of transplants for any organ type were of non-resident aliens
• special inquiries made by the Secretary of the Department of Health and Human Services (HHS)

The MPSC reviews and approves a tentative schedule of site visits for each year of the current OPTN contract. The schedule is adjusted as needed, based on the factors listed above, or to accommodate other meetings and staff activity in a given region of the country.

Site visits for both program centers and OPOs begin with an in-house review of the organization's compliance rate for forms submission deadlines, followed by an on-site review of medical records or donor records, and an interview with the program coordinator or other program personnel.

On-site Review of Medical and Donor Records

All liver and heart transplant programs and OPOs are visited at least once every three years. Institutions may be visited more frequently if circumstances warrant. For transplant programs, the focus of these visits is to verify medical records for compliance with listing criteria, medical urgency status justification, and other OPTN policies. For OPOs, the focus is to verify compliance with OPTN policies and donor records.

For every on-site audit, a standard report is prepared covering the following:

• introduction
• program background
• review methodology
• compliance with forms submission requirements
• review of data
• patient or donor data
• conclusions and recommendations

Reports are completed within three to four weeks of a site visit.

A letter sent with the report includes an executive summary of findings and recommendations and a date by which a response from the center is expected. The response allows the member to comment on the review process and submit any information it had agreed to provide.

Responses, including material that was not available during the audit (e.g., lab test results, nursing notes, donor records) are attached to the report. Issues of noncompliance are presented to the policy compliance subcommittee of the MPSC for further review and disposition. If a response to the report is not received in a timely manner, the Evaluation and Quality Department sends a letter to the center indicating that the report, its findings, and the center's failure to respond, is being sent to the MPSC.

Evaluation and Quality auditors use a scorecard to assess and summarize the administrative and clinical review of every reviewed program. The clinical score evaluates whether the program can verify all the urgent status listings with the appropriate medical record documentation.  The administrative score evaluates the center's performance with the following five factors: prompt removal of patient from urgent status listing after transplant or death, missing justification forms, availability of requested medical records on-site, submission of late justification forms, and errors on justification forms.

The scorecard provides a snapshot view of each center's compliance performance, which allows for a uniform comparison among centers.  Scorecard information is included in the auditors report.

Blinded scorecard information is forwarded to the MPSC on a quarterly basis for its review. Each reviewed program during the quarter is summarized and compared with other reviewed programs. The MPSC may request further information for a particular center based upon the scorecard summary.

The tables below summarize the scorecard compliance rates by program from October 2000 to October 2001, and October 2001 to July 2002.

Interview with Program Personnel

In addition to reviewing medical records, Evaluation and Quality auditors meet with facility staff upon arrival for a review, usually with the program coordinator. The auditor explains the review process and the approximate amount of time it will take to complete.  During the interview, the auditor and program staff will:

• discuss any records requested that are not available for review
• obtain any outstanding justification forms prior to the review
• review the "One-Year Compliance" and "Overall Data Submission Rates Forms" status for the program
• verify site personnel
• provide a UNOS membership audit with instructions and directions for updating program information and returning it to the Membership Department

For OPO reviews, the auditor reviews issues that are specific to OPOs such as organ packaging supplies and requires a demonstration of organ packaging from the OPO. The auditor also reviews how the OPO conducts public and professional education and how the OPO addresses multicultural issues within their respective regions.

Audits of Kidney Programs

Kidney programs are different from liver and heart programs in that kidney allocation is determined, in part, by the amount of time the patient has been waiting, not the severity of the patient's illness.

The Evaluation and Quality Department is currently working with the OPTN Kidney Pancreas Committee on various proposals to conduct renal audits.

For example, staff have been verifying patients' dialysis status, with patient forms, to determine whether kidney programs are providing accurate information. Research staff have been crosschecking patients' dialysis status with the patients' information entered on the waiting list and information submitted on the Transplant Candidate Registration (TCR) and Transplant Recipient Registration (TRR) Forms. The TRR requires the date of dialysis initiation. The TCR is completed at the time of listing, and the TRR is completed at the time of transplant.

The Research staff assessed all the adult patients added to the kidney waiting list between the period of August 1, 2000 to July 31, 2001. During that period, 21,984 adult patients were listed on the kidney waitlist and 18,036 patients indicated they were on dialysis at the time of listing. The patients' dialysis status was verified with the information entered on the waiting list and with the TCR and TRR Forms. The staff found that 96 percent of the kidney programs submitted accurate information regarding a patient's dialysis status. The other 4 percent may require an on-site review. This type of internal audit and analysis indicates which programs may potentially require on-site reviews.

At present, the staff is continuing to summarize and analyze data, such as the analysis of patients on dialysis, for the prospective of conducting on-site renal audits.

---

¹ As of July 2002.

² At the date of this publication, all liver and heart justification forms are submitted online.

³ Effective date of the new data submission standards in Policy 7.8.1.