* Context.-Newborns are often screened prior to discharge for hyperbilirubinemia. Transcutaneous bilirubin analyzers, such as the BiliChek, are promoted as screening tools, but it is unclear whether they also function well as monitoring devices. Newborns on home phototherapy require frequent determinations of serum bilirubin levels to monitor therapy effects. A transcutaneous bilirubin analyzer would be helpful to limit blood draws and enhance staff efficiency. We evaluated the accuracy of the BiliChek analyzer in this setting.
Objective.-Is the BiliChek sufficiently accurate to monitor the effectiveness of home phototherapy and establish when to terminate therapy?
Design.-Paired serum bilirubin results and results from the BiliChek were obtained from newborns on home phototherapy during daily home health care visits.
Results.-The BiliChek demonstrates a negative bias (mean bias, -1.71 mg/dL; 95% confidence interval, -1.89 to -1.52 mg/dL) compared with serum bilirubin values. This bias worsens as the serum bilirubin level rises. If a value of 14 mg/dL or less obtained using the BiliChek had been used as the cutoff for termination of phototherapy, 45% of newborns would have had therapy terminated prematurely. If, knowing the negative bias of the BiliChek, the cutoff for termination of therapy was set at less than or equal to 11 mg/dL, then 29% of newborns would have had therapy terminated prematurely.
Conclusions.-The values obtained using the BiliChek, compared to serum bilirubin values, have a negative bias that worsens at the higher bilirubin levels expected in newborns at home on phototherapy. The BiliChek does not provide sufficient accuracy to be utilized to monitor newborns on home phototherapy or to ascertain when to discontinue such therapy.
(Arch Pathol Lab Med. 2008;132:684-689)
Hyperbilirubinemia, with the potential risk of kernicterus, has recently gained attention as new cases of this preventable disorder were reported in the early 2000s.1 Guidelines for the management of hyperbilirubinemic newborns have been established by the American Academy of Pediatrics.2,3 In our community, many of these newborns are discharged to receive home phototherapy and require measurement of bilirubin levels to monitor the effects of therapy and to ascertain when to terminate treatment. The serum bilirubin level at which to terminate home therapy is not provided in guidelines, but for patients readmitted to the hospital and treated for hyperbilirubinemia, the recommendation is to discontinue phototherapy once the serum bilirubin level falls below 13 to 14 mg/dL.2 Most of our clinicians use 12 to 14 mg/dL, along with a general evaluation of the newborn's status.
The BiliChek (Respironics, Marietta, Ga) is a device that measures bilirubin transcutaneously. Demonstrated to be useful to screen for hyperbilirubinemia in newborn nurseries, 4-8 it is also claimed by the company to be accurate enough to use as a monitoring device in newborns undergoing phototherapy.8 As newborns on home therapy may require several days of treatment, a method of measuring bilirubin that does not entail a venous or heel puncture and that can give instant results would be very useful. We evaluated the BiliChek in this population to ascertain whether it was sufficiently accurate to allow for monitoring of patients and determining when to terminate therapy.
MATERIALS AND METHODS
The BiliChek devices were acquired (on a trial basis) by Children's Hospital Home Health Care Services (Omaha, Neb) in March 2006. Training of home health care nurses and respiratory therapists was performed. It consisted of a 2-day session of hands-on training by the manufacturer's representative and observational training at the Methodist Hospital Newborn Nursery (Omaha, Neb) teaching site to observe technicians performing a bilirubin level test with the BiliChek on newborns on the day of their discharge. Methodist Nursery has been using the BiliChek device for more than 2 years as a screening tool. A check-off sheet was used to evaluate all trainees in the operational steps of the BiliChek device (including error codes, battery removal, criteria when not to use device), as well as the step-by-step process of using the device. This was followed by hands-on training with the device and testing on each other. After all trainees were checked off as competent in the use of the device, a 2-month trial to determine variability among staff was performed to identify any retraining needs.
Three devices were obtained (1 device was replaced due to performance failure-a broken piece of plastic).
Home health care nurses and respiratory therapists were assigned to visit hyperbilirubinemic newborns for whom phototherapy was ordered by their physicians on their first day at home, and to draw a blood sample for serum bilirubin measurement, perform a bilirubin level test using the BiliChek, set up phototherapy (either a Wallaby III system and/or PEP unit [an Ultra BiliLight device]), and weigh and evaluate the newborn. Only newborns on the Wallaby system were tested with the BiliChek. All newborns were tested on the forehead. Specimens for serum bilirubin testing were transported to Children's Hospital for processing. The clinician returned each subsequent day to evaluate the newborn, weigh the newborn, draw a blood sample for serum bilirubin testing, and obtain a BiliChek bilirubin level. Each newborn received a daily serum bilirubin test for 1 to 4 days on average. When the serum bilirubin value was approximately 12 to 14 mg/dL, and the newborn was doing well clinically, the phototherapy was terminated by the ordering physician. Upon discontinuation of home phototherapy by the physician (based upon the serum bilirubin result), the clinician would return to the patient's home to remove the phototherapy equipment.
Specimens for serum bilirubin testing were collected in microtainers that protected the specimens from light. Upon arrival in the laboratory, the specimens were spun and analyzed on a Fusion 5,1 (Vitros/Ektachem; Ortho-Clinical Diagnostics), using a 10-�L sample drop volume on the dry slide technology. Both the unconjugated (Bu) and conjugated (Bc) bilirubin were measured from the same slide. The test reaction is an end point colorimetric, dual-wavelength reaction that employs the unique spectral characteristics of Bu and Bc, with readings at both 400 and 460 nm. Results are reported in milligrams per deciliter. Per routine, quality control is run daily and monitored for deviations from the expected range. Calibrations are performed at least every 6 months or whenever deemed necessary due to implementation of a new lot number, after certain service procedures, or in the event of unacceptable quality control deviation.
The data collected listed each patient's identification number, which BiliChek device was used, which staff member performed the BiliChek test, the hours of age of the newborn, the serum bilirubin level, and the bilirubin level obtained using the Bili- Chek. Allowing for a several-month learning curve, only data obtained since June 1, 2006 were used. EP Method Validator was the program used to obtain statistical values. Bias plots and Deming regression were utilized to compare the 2 methods.9
RESULTS
Two hundred nine newborns were included in the study. Only very rarely would a newborn have a significant increase in bilirubin levels after arriving home; most were already at or near their peak bilirubin level, and most of the values demonstrated a daily decline. Rarely would a newborn have a mild increase in bilirubin after several days of therapy. The newborns were greater than 34 weeks' gestation. Underlying risk factors for hyperbilirubinemia were unknown, as was racial distribution. Most of the newborns were breast-fed, but these data were not collected and many received various amounts of fluid and formula supplementation. The average age of the newborns at the time of the first home visit was 94.6 hours (median, 90.5 hours; range, 39.9-530.4 hours).
The data were subjected to Grubbs analysis for outliers. Some differences between the bilirubin levels obtained using the BiliChek and the serum bilirubin levels appeared extreme (up to 8.9 mg/dL), but there was a continuous range of differences and therefore these apparent extreme differences could not be dismissed as outliers.
A difference plot using all data points (N = 477) demonstrated that the BiliChek has a mean bias of -1.71 mg/ dL (95% confidence interval [CI], -1.89 to -1.52 mg/dL) compared with the serum bilirubin level (Figure 1). Deming regression analysis demonstrated a slope of 1.204 (95% CI, 1.083-1.324) with the intercept -4.83 (95% CI, -6.67 to -3.00) (Figure 2). The correlation coefficient (r) was 0.662.
According to its product literature,8 Respironics performed correlation studies of the BiliChek on newborns with bilirubin levels that were relatively low compared with the newborns we evaluated. Only 3.3% of the newborns in the Respironics studies had bilirubin values greater than 14 mg/dL, and only 11.8% of these newborns had values greater that 12 mg/dL. Other studies found in the literature also evaluated data on serum bilirubin values that were predominantly 13 to 15 mg/dL or less.4-6 In an article by Bhutani et al,4 the range of bilirubin values was 0.2 to 18.2 mg/dL, but only 1.1% of the values were greater than 15 mg/dL and only 21.7% were greater than 10 mg/dL. The exception is the article by Engle et al,7 in which populations had elevated levels, with 31% of Hispanic newborns and 9% of non-Hispanic white newborns having values greater than 15 mg/dL. Our population was similar to that of Engle et al in this respect. To evaluate whether the level of bilirubin affected the accuracy of the BiliChek, we evaluated bias plots for serum bilirubin levels less than 12.9 mg/dL, from 13 to 14.9 mg/dL, and greater than 15 mg/dL. We observed that at levels less than 12.9 mg/dL (n = 66; 14% of all values), the bias was less at -0.897 mg/dL (95% CI, -1.26 to -0.534 mg/dL). At levels between 13 and 14.9 mg/dL (n = 137; 29.3% of all values), the bias was -1.41 mg/dL (95% CI, -1.74 to -1.08 mg/dL). At levels greater than 15 mg/dL (n = 268; 56.7% of all values), the bias was -1.99 mg/dL (95% CI, -2.24 to -1.74 mg/dL). Thus, we observed a significant worsening of negative bias as serum bilirubin levels rose (Figure 3).
One of our goals was to ascertain whether the BiliChek was accurate enough to be used to monitor and terminate phototherapy. Home health care staff proposed the following algorithm and question. On day 1, both a serum and a BiliChek bilirubin value would be obtained. If the newborn was older than 96 hours of age, and if there had been at least 2 declining bilirubin levels via BiliChek, could the final decision to withdraw phototherapy be made based on a subsequent further decrease in the BiliChek bilirubin level alone, without having to repeat a serum bilirubin? This would mean fewer venipunctures or capillary draws for serum bilirubin levels. The staff also would be able to finalize care of the patient in a single visit, removing the equipment during the same visit at which they obtained the final bilirubin level using the BiliChek. This would improve efficiency by eliminating delivery of a specimen for serum bilirubin determination to the laboratory, awaiting the results, contacting the physician, and then returning to the home to remove equipment.
We therefore evaluated the third consecutive decreasing serum bilirubin values versus the BiliChek bilirubin levels (n = 79) and found that the bias of the BiliChek was -1.78 mg/dL (95% CI, -2.18 to -1.39 mg/dL). For study purposes, we chose a bilirubin level of 14 mg/dL or less as the decision point to terminate phototherapy. If we had chosen to terminate phototherapy when the bilirubin level obtained using the BiliChek was 14 mg/dL or less (n = 62), 45% of the newborns would have had phototherapy terminated before their serum bilirubin values were less than or equal to 14 mg/dL. Most would have had serum bilirubin values in the 14 to 15 mg/dL range, but some were in the 16 mg/dL range, with rarely a newborn having a serum bilirubin value as high as 17.5 mg/dL. Also, by using a BiliChek bilirubin level cutoff of 14 mg/dL, 12% of newborns (n = 17) would have continued to receive unnecessary therapy.
In an attempt to correct for the fact that the BiliChek has a negative bias, we re-evaluated the data based on a lower cutoff value, a BiliChek bilirubin level of 11 mg/dL or less (n = 24). Using this decision point, 29% of the newborns would have had therapy terminated before their serum bilirubin levels were less than 14 mg/dL, while 33% (n = 55) would have received 1 or 2 days of unnecessary therapy (Figure 4).
In an attempt to rule out user error, the first 2 months of data were evaluated. The overall bias (n = 89) was -1.39 mg/dL (CI, -1.77 to -1.01 mg/dL). The distribution of serum bilirubin levels was comparable to the overall distribution in the post training period data (21.3% less than 12.9 mg/dL, 29.2% between 13 and 14.9 mg/dL, and 49.5% greater than 15 mg/dL). This bias (-1.39 mg/dL) is somewhat better compared with that obtained when the training period data are eliminated (-1.71 mg/dL), possibly secondary to fewer data points.
We then analyzed the data beginning after the training period by user, to look for single user error that might account for the negative bias identified. There was no pattern evident when we compared the number of times a clinician used the BiliChek with the average serum bilirubin versus the BiliChek bilirubin difference. In other words, familiarity with the BiliChek did not lessen the bias (Figure 5).
Finally, we evaluated the impact of age of the newborn on the results obtained using the BiliChek. The BiliChek literature8 states that the instrument is designed to evaluate newborns from 0 to 20 days of age. Our age span was from 1.6 to 13.7 days (with a single outlier at 22.8 days with a difference of -1.9 mg/dL; not included in the data for Figure 6). The bias did not change based on the newborn's age (Figure 6).
COMMENT
The BiliChek is a device designed for screening newborns for hyperbilirubinemia. The BiliChek product literature8 indicates a bias of /- 1.5 mg/dL. There are numerous studies4-7 that evaluate the BiliChek as a screening device. During screening, most bilirubin levels are relatively low, within the range in which (based on our data) the BiliChek demonstrates improved accuracy compared to higher bilirubin levels. This permits establishment of BiliChek bilirubin cutoff values beyond which obtaining a serum level is prompted for further evaluation of the newborn.
Utilizing an analyzer for a completely different purpose than intended (screening vs monitoring) opens the door for great discrepancies in its performance.10 However, the literature for BiliChek also indicates8 that the device is accurate enough to use for monitoring newborns receiving phototherapy. The BiliChek study obtained a correlation factor (r) of 0.87 to 92; no bias plot or data are provided. Of interest, however, of the patients on phototherapy who were monitored by the BiliChek manufacturer,8 only 3.3% had bilirubin levels greater than 14 mg/dL.
Only a few other studies have evaluated the BiliChek as a monitoring device. One is the study by Engle et al,7 who evaluated newborns on phototherapy. As in our study, they had newborns with elevated bilirubins, most expected to be at least 15 mg/dL. They also uncovered a significant negative bias that limited the ability of the BiliChek to be utilized as a monitoring tool. Both Hispanic newborns (31%) and non-Hispanic white newborns (9%) in their study had serum bilirubins greater than 15 mg/dL; they report a similar bias irrespective of race. While we cannot confirm that race does not play a role in this bias because we do not have race data on our newborn population, we suspect that there is a nonlinear bias that worsens as bilirubin levels increase.
In addition, all studies identified to date were performed in newborn nurseries or outpatient pediatric clinics. 3-8 No studies evaluating the device in the home health care setting could be found. The home health care setting is a very different environment with many more opportunities for preanalytic error than in the newborn nursery. It is possible that this setting and the resultant error are contributing to the bias we are observing.
Although we initially considered user error as a potential confounder of our data, review of the data indicates no correlation between familiarity with the BiliChek and the bias obtained.
Potential problems with this study are several. First, the home health care environment itself, with other children, distractions, etc, may have contributed to our bias. Second, our comparison method was the Fusion 5,1 analyzer, and this could have been inaccurate, although based on quality control data and other monitors, we could identify no aberrations. Third, we do not know the racial distribution of our patients. Finally, we did not evaluate the precision of the BiliChek in this study.
CONCLUSIONS
It would be very useful to have a transcutaneous, point-of- care method to measure bilirubin in the newborn at home on phototherapy and to monitor treatment. However, in this setting, newborns have expectedly high bilirubin levels. Our data show that the negative bias of the BiliChek worsens as the bilirubin level increases. The BiliChek does not offer adequate accuracy to allow it to be used to monitor newborns on home phototherapy or to ascertain when to discontinue therapy.
[Reference]
References
1. Joint Commission on Accreditation of Healthcare Organizations (JCAHO). Kernicterus threatens healthy newborns. Sentinel Event Alert 2001;18(18):1-2. Available at: http://www.jointcommission.org/SentinelEvents/SentinelEventAlert/ sea=18.htm. Accessed September 30, 2007.
2. Subcommittee on Hyperbilirubinemia, American Academy of Pediatrics, Clinical Practice Guidelines. Management of hyperbilirubinemia in the newborn infant 35 or more weeks of gestation. Pediatrics. 2004;114:297-316.
3. Stanley IP, Chung M, Kulig J, et al; and Subcommittee on Hyperbilirubinemia. Evidence-based review of important issues concerning neonatal hyperbilirubinemia. Pediatrics. 2004;114:e130-e153.
4. Bhutani VK, Gourley GR, Adler S, Kreamer B, Dalin C, Johnson LH. Noninvasive measurement of total serum bilirubin in a multiracial predischarge newborn population to assess the risk of severe hyperbilirubinemia. Pediatrics. 2000; 106(2):E17.
5. Dominguez T, Bukovitz M, Swartz H, Buckellew M. Implementation of the new transcutaneous bilirubinometer, BiliChek�, in the maternity ward and pediatric outpatient clinic. The Society of Armed Forces Medical Laboratory Scientists (SAFMLS) Society Scope. Summer 2005;8-11. Available at: http://www.safmls. org/Scopes/Scope%20-%202005%20Summer.pdf. Accessed September 30, 2007.
6. Rubaltelli FF, Gourley GR, Loskamp N, et al. Transcutaneous bilirubin measurement: a multicenter evaluation of a new device. Pediatrics. 2001;107:1264- 1271.
7. Engle WD, Jackson GL, Sendelbach D, Manning D, Frawley WH. Assessment of a transcutaneous device in the evaluation of neonatal hyperbilirubinemia in a primarily Hispanic population. Pediatrics. 2002;110:61-67.
8. BiliChek Noninvasive Bilirubin Analyzer, User Instruction Manual, Respironics, Inc.
9. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307-310.
10. Schumacher RC. Transcutaneous bilirubinometry and diagnostic tests: ''the right job for the tool.'' Pediatrics. 2002;110:407-408.
[Author Affiliation]
Christine A. Reyes, MD; Donald R. Stednitz, BA, RRT-NPS, AE-C; Carol Hahn, MT(ASCP); Kelly D. Mutchie, PharmD; Steven R. McCullough, MT(ASCP); Kent Kronberg, MD
Accepted for publication October 12, 2007.
From the Department of Pathology (Dr Reyes, Ms Hahn, and Mr McCullough), Children's Home Health Care (Mr Stednitz and Dr Mutchie), and Children's Physicians Eagle Run (Dr Kronberg), Children's Hospital, Omaha, Neb.
The authors have no relevant financial interest in the products or companies described in this article.
Reprints: Christine A. Reyes, MD, Department of Pathology, Children's Hospital, 8200 Dodge St, Omaha, NE 68114 (e-mail: creyes@ chsomaha.org).
