Birth of a New Methodology

We’ve added death rates, nursing, and other ways to look at care

By Avery Comarow

In July, America’s Best Hospitals turned 18. From the very first, U.S. News has ranked hospitals in pediatrics, identifying medical centers that excel at helping the sickest young patients. We’ve heard reassuringly few complaints through the years that one or another of the ranked hospitals might not merit such distinction.

Still, improving the usefulness and relevance of the rankings in this important specialty has been a long-standing goal. Mortality data and other information factored into various adult specialty rankings cannot be obtained for children’s hospitals. In pediatrics, therefore, the rankings have always relied solely on a facility’s reputation among an annual sampling of board-certified physicians in various pediatric specialties. We’ve long wanted to move beyond reputation.

We also want to rank children’s hospitals in subspecialties such as heart, cancer, and respiratory disorders, as we do for adult hospitals. But pediatric experts are currently wrangling over how best to define, collect, and verify the kind of data that have real meaning as measures of the quality of pediatric centers. It’s no easy assignment. Coming to agreement might take another three to five years.

The rankings shown here signal that U.S. News chose not to wait. Instead, we enlisted our Best Hospitals contractor, RTI International, in devising an enhanced approach. It is the first in a series of planned improvements to the pediatrics rankings methodology that will be introduced in stages. The second stage is planned for next spring.

A medical facility was considered eligible for this year’s rankings if classified by the National Association for Children’s Hospitals and Related Institutions (nachri) as a freestanding children’s hospital or as a children’s “hospital within a hospital”—a large, multidisciplinary pediatric service within a medical center. As of March 1, 2007, 122 hospitals met this standard.

Reputation has been supplemented with data obtained by directly surveying the 122 eligible U.S. children’s hospitals, 113 of which responded. The rankings, like their adult Best Hospitals counterparts, now reflect a three-part mix of reputation, death rate, and care-related factors such as volume, nursing care, advanced technology, and recognition by outside organizations. (Details are described in the glossary
below.) The three parts break down as follows:

Reputation. For the 2007 rankings, 200 board-certified pediatricians, adolescent medicine specialists, and (for the first time) neonatologists were randomly selected from the American Medical Association’s Master file of more than 850,000 U.S. physicians. Those doctors were mailed a survey form asking them to name five hospitals they consider best for a child in need of the highest level of care; they were instructed not to consider geography or cost and not to name hospitals where they practice. The “reputation” column in the rankings indicates the percentage of responding physicians in 2005, 2006, and 2007 who named the ranked hospitals. Slightly less than half of the physicians surveyed over the three years provided responses. Reputation counted as two thirds of each hospital’s overall score.

Death rate. Hospitals were asked to report their patient volume and number of inpatient deaths in the most recent 12 months for three procedures: a complex heart repair (of a defect called tetrology of Fallot), bone marrow transplant, and removal of a cancerous brain tumor. A hospital could receive up to six points—zero, one, or two for each of the three procedures—depending on the number of procedures and the death rate. No adjustments were made for patient severity. Death rate counted as one sixth of the score.

Care-related factors. We asked hospitals for the total number of inpatients treated in the most recent year (not counting healthy newborns), the total number of nurses, and availability of the kinds of advanced care expected from a “best” pediatric hospital, such as key imaging technologies and a formal palliative care program. We also weighed certification as a Nurse Magnet hospital by the American Nurse Credentialing Center (ancc) and designation by the Foundation for the Accreditation of Cellular Therapy (fact) for cell transplantation. Care-related factors counted as one sixth of the score.

GLOSSARY

U.S. News Score
Summarizes overall quality of inpatient care. Reputation represents two thirds of each hospital’s score, mortality one sixth, and a combination of overall hospital volume, two nursing measures, and three types of advanced care one sixth. A score of 100 is assigned to the top-ranked hospital.

Reputation (%)
Percentage of pediatricians, adolescent-medicine specialists, and (starting in 2007) neonatologists responding to U.S. News surveys in 2005, 2006, and 2007 who listed the hospital as among the five best in their specialty for particularly challenging patients.

Mortality
Maximum of 6 points awarded—0, 1, or 2 points in each of three key procedures based on inpatient death rates during 2006 or the most recent 12-month period with available data. The procedures: repair of tetralogy of Fallot (a combination of four congenital heart defects), removal of a cancerous brain tumor, and bone marrow transplantation. To receive points, hospitals had to meet specific volume and mortality requirements.

Discharges (all patients)
The total number of inpatients, excluding healthy newborns but including neonatal discharges, in 2006 or the latest year for which data are available.

Nurse staffing
Indicates relative balance of nurses to inpatients; higher is better. Nurses must be on-staff R.N.’s, not temps or workers provided by agencies. The number shown is based on full-time nurse equivalents and an adjusted daily average of inpatient volume.

Nurse Magnet hospital
A designation indicating that as of April 1, 2007, the hospital satisfied standards set by the American Nurses Credentialing Center for nursing excellence.

Advanced care
A mix of three measures—the key technologies indicated below, a palliative-care program, and fact accreditation for cell therapy—that help define a top children’s hospital.

Technology (of 5)
One point is awarded for each of five technology services available on site (half a point for a service that is off site but nearby): image-guided radiation therapy, intensity-modulated radiation therapy, shaped-beam radiation, positron emission tomography, and stereotactic radiosurgery.

Palliative care program
An organized, staffed program directed at children who are terminally ill or have a condition that severely compromises their quality of life, such as chronic pain. The intent is to minimize pain and discomfort, support children and their families emotionally and spiritually, and help with financial guidance and needed social services. An active program received 1 point; a less-active program received half a point.

FACT accreditation
Indicates whether as of April 1, 2007, the hospital met standards set by the Foundation for the Accreditation of Cellular Therapy for transplantation of cells, both from another person and from the patient, in order to treat cancer.