Sometimes labs get slick with calling something validated because people assume it means FDA validated. It does not. A lab can do an internal study to determine the sensitivity and specificity of a test, write up a report and call it internally validated without ever needing to involve outside parties. Internally validated tests can have value, but it is difficult to ascertain without getting into the very gory details of the data, a lot of which is unpublished. Likewise, labs can develop their own “in house laboratory criteria” for FDA validated tests.
Some doctors send their samples to “Lyme specialty” laboratories based upon the belief that the ELISAs and Western blots performed there will be more sensitive. The 2014 paper mentioned in the previous post (Fallon 2014) looked at how likely four labs were to correctly perform Lyme diagnostics. They used one university reference lab, one commercial lab, and two Lyme specialty labs. Importantly, this study also included the in-house validated tests (NOT FDA validated tests) performed at the specialty lab, which included an ELISA for a different Lyme target (C6), used alone or in combination with the standard ELISAs or western blots using different criteria than the CDC recommended tests.
Specialty Lab A defined an IgM western blot as positive if at least two of the following bands were present: 23, 39, 41, 83/93 (these numbers refer to the size of the targets), and defined an IgG western blot as positive if at least 3 of the following bands were present: 20, 23, 31, 34, 35, 39, 83/93. Specialty Lab B defined an IgM western blot as positive if at least two of the following bands were present: 23-25, 31, 34, 39, 41, 83/93 (these numbers refer to the size of the targets), and defined an IgG western blot as positive if at least 2 of the following bands were present: 23-25, 31, 34, 39, 41, 83/93.
Here’s how the results shook out:
Standard IgM/IgG ELISA: University reference lab: 62.2% positive; commercial and specialty lab: 67.6% positive. The commercial and specialty labs both ran samples in pairs. Pairs in which one is positive and the other is not are called discordant. The commercial lab had 12 discordant pairs, specialty lab A had 14 discordant pairs, and specialty lab B had 8 discordant pairs.
C6 ELISA: Specialty lab A, 67.6% positive; specialty lab B, 62.6% positive, not run in pairs.
IgM Western blot using CDC criteria: university reference lab: 21.6% positive; commercial lab: 16.2% positive, 8 discordant pairs; specialty lab A: 2.7% positive, 7 discordant pairs; specialty lab B: 43.2% positive, 10 discordant pairs.
IgM Western blot using inhouse lab criteria: specialty lab A: 2.7% positive, 7 discordant pairs; specialty lab B: 62.2% positive, 15 discordant pairs.
IgG Western blot using CDC criteria: university reference lab: 56.8% positive; commercial lab: 43.2% positive, 5 discordant pairs; specialty lab A: 43.2% positive, 5 discordant pairs; specialty lab B: 48.6% positive, 3 discordant pairs.
IgG Western blot using inhouse lab criteria: specialty lab A: 37.8% positive, 7 discordant pairs; specialty lab B: 70.3% positive, 7 discordant pairs.
Two tier: standard IgM/IgG ELISA and IgG western blot (CDC criteria): university reference lab: 48.6% positive; commercial lab: 40.5% positive, 3 discordant pairs; specialty lab A: 37.8% positive, 6 discordant pairs; specialty lab B: 43.2% positive, 6 discordant pairs.
Two tier: C6 ELISA and IgG western blot (CDC criteria): specialty lab A: 40.5% positive, not run in pairs; specialty lab B: 45.9% positive, not run in pairs.
Two tier: Standard ELISA and C6 ELISA: specialty lab A: 59.5% positive, not run in pairs; specialty lab B: 48.6% positive, not run in pairs.
When running a standardized test (the CDC recommended ELISA and western blots), Lyme specialty labs find the same amount of positives as the commercial lab using the ELISA; find much fewer or much more positives using the IgM blot; and find the same amount of positives, or very close to the same, as the commercial lab for the IgG blot. When running the two recommended two tier test, they find almost the same amount of positives as the commercial lab.
When running the C6 ELISA, they find similar results to when they run the IgM/IgG ELISA. When they run their own criteria IgM blots, one lab got the same results as CDC criteria, and one found almost 20% more positives. When running their own criteria IgG blots, one lab found fewer positives than the CDC criteria and the other found almost 22% more positives. (Both of these large increases were found at the lab which had the least stringent criteria.)
Both labs found had a small increase in the number of positives found when running the two tier test with C6 ELISA and their own western blots. When they used only ELISAs (C6 and standard), one lab saw almost a 20% increase, while the other saw less than 3% increase in positives. I am not surprised by increases when using two ELISA tests together – the specific reason for using a western blot as the second step is that ELISAs frequently report false positives.
The results found on standardized tests are heavily dependent on the lab. However, the Lyme specialty labs do not always find more positives. When you look at the CDC two tier test, the University reference lab found the most positives, with the commercial lab and Lyme specialty labs finding positives in the same range. The two tier test using the C6 ELISA and in house western blots found positives slightly higher than the CDC two tier test when run by Lyme specialty labs. By contrast, their two tier, two ELISA test (C6 and standard) offers huge gains at one lab but not the other.
The take home from this study is this: When running standardized tests, the results are very specific to the lab, but when diagnostic two tier testing is performed in accordance with CDC guidelines, Lyme specialty labs have similar results as commercial labs. Lyme specialty labs actually find fewer positives with the CDC two tier test than the universe reference lab, which had the highest degree of accuracy in this study.
ELISA and western blots are used together to correct for ELISA finding false positives. When using in house tests and criteria, the Lyme specialty labs had a very modest increase (less than 3%) in positives. When using two ELISA test, one lab showed an increase in positivity of less than 3%, while the other lab showed almost a 20% increase. Please note that because ELISAs find false positives, using two ELISAs means you will find more false positives. The increases sometimes seen when using the blot tests alone are not because the Lyme specialty labs run the tests more accurately. They are because the labs have their own in house criteria which drop the bar significantly.
Also, discordancy is the death knell of diagnostics. And while commercial labs have a fair amount of discordancy (about 8% using the two tier test), both Lyme labs tested here have double that discordancy when running the diagnostic two tier test.
References:
Brian A. Fallon, Martina Pavlicova, Samantha W. Coffino, and Carl Brenner. A Comparison of Lyme Disease Serologic Test Results From 4 Laboratories in Patients With Persistent Symptoms After Antibiotic Treatment Comparison of Serologic Test Results. Clinical Infectious Diseases 2014: 59 (15 December) , 1705-1710.
Andrea T. Borchers, Carl L. Keen, Arthur C. Huntley, M. Eric Gershwin. Lyme disease: A rigorous review of diagnostic criteria and treatment. Journal of Autoimmunity 57 (2015) 82-115.