|Home | About | Journals | Submit | Contact Us | Français|
Data are accumulating on the performance of enzyme immunoassays (EIAs) for the detection of herpes simplex virus type 2 (HSV-2) infection in North America and Europe, but little is known about their performance in other populations. Nine test kits were evaluated with 330 serum samples from sub-Saharan Africa. The tests were first compared to the monoclonal antibody (MAb) EIA (Central Public Health Laboratory, London, United Kingdom). Samples that gave discordant results in the MAb EIA and in the three tests that performed best compared to the MAb EIA were tested by Western blotting (University of Washington, Seattle). A random sample of concordant samples was also tested, and the sensitivities and specificities of the different tests were calculated, taking into account this sampling strategy. The sensitivities of the tests ranged from 86 to 100%; the specificities ranged from 47 to 99%. The tests that performed best were the Gull Premier EIA (sensitivity, 86.3%; specificity, 97.6%) and the Kalon Biological (sensitivity, 92.3%; specificity, 97.7%) and Biokit (sensitivity, 86.7%; specificity, 92.6%) tests. It cannot be assumed that enzyme immunoassays for the detection of HSV-2 infection that perform well in industrialized countries will perform equally well in other populations.
Herpes simplex virus type 2 (HSV-2) is the most common cause of genital ulcer disease in industrialized countries. In recent years the importance of genital herpes has been increasingly recognized in developing countries as well. In several population-based studies in sub-Saharan Africa, high rates of infection with HSV-2 have been detected (10, 12, 13, 16). HSV-2 infection is strongly associated with human immunodeficiency virus (HIV) infection, and it is thought that HSV-2 infection plays an important role in the spread of HIV in sub-Saharan Africa (4, 15, 16).
Although genital herpes was first described in the 18th century, it was not until the 1960s that the distinction between HSV-1 and HSV-2 was discovered (6, 11). HSV-2 infection can be diagnosed by virus culture and DNA amplification methods, but these diagnostic techniques are complex, very expensive, and not suited for settings with limited resources or for use in large-scale epidemiological studies. Serological tests for the detection of specific antibodies against HSV-2 and HSV-1 were until recently restricted to research laboratories. There are several serological tests that are offered in academic or reference laboratory settings and that can be used to establish the performance of other tests. These tests include the Western blot developed at the University of Washington (Seattle), the immunodot enzyme assay for gG-2, and the monoclonal antibody-blocking enzyme immunoassay (MAb EIA) developed by the Central Public Health Laboratory (London, United Kingdom) (2). In the past few years several commercial tests have become available for the detection of antibodies to HSV-2. In 2001, three tests were approved by the U.S. Food and Drug Administration (FDA) for the diagnosis of HSV-2 infection, including the HerpeSelect (Focus Technologies, Inc., Cypress, Calif.), the HSV-1 and HSV-2 IgG differentiation immunoblot (Focus Technologies), and the POCkit HSV-2 (Diagnology, Ltd., Belfast, Northern Ireland). A fourth test had been approved by FDA, the Premier Type-Specific HSV-2 IgG EIA (Meridian, Cincinnati, Ohio; first developed by Gull Laboratories, Inc.), but this test is no longer available. These tests have all been evaluated in industrialized countries, i.e., North America and Western Europe. The sensitivities, as assessed against one or more of the “gold standard” tests for the detection of antibodies against HSV-2, ranged between 93 and 100%; the specificities ranged between 95 and 100% (2).
When planning a multicenter study on factors determining the differential spread of HIV in four African cities, we conducted a small preliminary study to test several commercially available tests for the detection of antibodies to HSV-2 (5). Although HerpeSelect performed well in studies in the United States, we found that it had an unacceptably low specificity when used on our sera from Africa (14). We therefore decided to test our samples with the Gull Premier Type-Specific IgG EIA and to conduct a larger-scale study later of the sensitivities and specificities of commercial EIAs with sera from African populations. We present here the results of the large-scale evaluation.
The performance of commercial HSV-2 serological tests was assessed using serum samples that were collected in a multicenter study on factors determining the differential spread of HIV in four African cities. The objective of this study was to gain better insights into the dynamics of the HIV epidemics in different regions of sub-Saharan Africa. The study was conducted in two cities with a high prevalence of HIV infection (Kisumu in Kenya and Ndola in Zambia) and in two cities with a relatively low prevalence of HIV infection (Cotonou in Benin and Yaoundé in Cameroon). In each of the four cities, representative samples of about 1,000 women and 1,000 men aged 15 to 49 years were obtained from the general population. The methods of the multicenter study are described in detail elsewhere (5). In women the prevalence of HSV-2 infection, as assessed by the Gull Premier Type-Specific IgG EIA, ranged between 30% in Cotonou and 68% in Kisumu; in men it ranged between 12% in Cotonou and 36% in Ndola (16).
For the present study, 600 samples were selected from among the frozen serum samples that were still available from the multicenter study. The sampling scheme was designed in such a way that there would be enough power to measure the sensitivities and specificities in different subgroups, i.e., HIV-1-positive and HIV-1-negative subjects, men and women, East and West Africans, and older and younger age groups.
An internet search was done to identify commercially available EIAs for the detection of antibodies to HSV-2. Laboratories were contacted for additional information about their tests if the information on the internet was not detailed enough to make a preliminary assessment of the test. In a first phase, 90 samples were tested with all of the selected tests according to the manufacturers' instructions. Tests that gave a positivity of ≥90% were likely to have a low specificity and were excluded from further evaluation. The sensitivities and specificities of the different tests, including the Gull Premier Type-Specific IgG EIA, were first assessed against the MAb EIA developed at the Central Public Health Laboratory in London. The three tests that had the highest sensitivities and specificities compared to the MAb EIA were identified. Samples that gave discordant results on the four best tests, i.e., the three best commercial tests and the MAb EIA, were further tested by Western blotting at the University of Washington (1). In order to avoid bias in the assessment of the sensitivity and specificity (7), a random collection of positive concordant and negative concordant samples was also tested. The sensitivity and specificity of the different tests were calculated, taking into account this sampling strategy, according to the methods described by Hawkins et al. (8). An example of the calculation is provided in the appendix.
Thirteen tests were found via the internet (see Table Table1).1). Four tests gave more than 90% positives on the initial run of 90 samples, whereas the percentage positive samples for the other tests ranged between 51 and 82% (Table (Table1),1), so these were excluded from further evaluation. The tests that were excluded were the Captia HSV-2 IgG of Trinity Biotech and the tests of Wampole Laboratories, Genzyme Virotech, and Zeus Scientific. These tests all used crude antigen.
Of the 600 serum samples, only 330 could be tested by the MAb EIA. There was insufficient serum left of the samples from Cotonou to carry out the tests. Table Table22 gives the prevalence of HSV-2 infection as measured by the different tests and the sensitivities and specificities versus that of the MAb EIA. The three tests that performed best compared to the MAb EIA were the Gull Premier Type-Specific IgG EIA and the Kalon Biological and Biokit tests. Of the 330 samples, 160 (48.5%) were positive in all of these three tests and, in the MAb EIA, 57 samples (17.3%) were discordant and 113 (34.2%) were negative in all four tests. All of the 57 discordant samples were tested by Western blotting and, of these, 42 samples were found to be positive by Western blotting and 1 was indeterminate. Of the concordant positive samples, 23 were tested by Western blotting and all were found to be positive. Of the concordant negative samples, 20 were tested by Western blotting; 19 were determined to be negative, and 1 was indeterminate.
Table Table33 gives the sensitivities and specificities of the different tests with the Western blotting as the gold standard. Apart from the MAb EIA, the test with the highest sensitivity and specificity was the Kalon Biological test (sensitivity, 92.3% [95% confidence interval = 89 to 96%]; specificity, 97.7% [95% confidence interval = 92 to 100%]). Variations in sensitivity and specificity by city and HIV status are shown in Table Table44 for the different tests. In general, the specificity tended to be lowest in Kisumu analyses and lower in HIV-positive samples than in HIV-negative samples. Variations by age were also explored, but there was no clear pattern when younger age groups were compared to older age groups (data not shown). There were also no differences in test performance in men and women (data not shown).
Table Table55 shows the distribution of the ratios of the sample optical densities (ODs) to the standard OD for samples that gave false-positive results. For this analysis, a true-positive sample was defined as a sample that was positive in the four best tests or in the Western blot, and a true-negative sample was negative in all four tests or in the Western blot. Most false-positive results had low ratios, but at least one in three false-positive results obtained with the HerpeSelect, Captia HSV-2 Type-Specific IgG, and DiaSorin tests had ratios of the OD values of >2.
We found large variations in the performance of the different commercially available EIAs for the detection of antibodies against HSV-2. Four of the thirteen tests we selected for assessment were not evaluated further after an initial run of 90 samples revealed a very high false positivity rate. These four tests used crude antigen as antigen, and problems with the tests from Wampole Laboratories and Zeus Scientific have been reported previously (2). Among the remaining test kits, the sensitivities ranged between 83.3 and 100%. Differences in specificity were more marked, with estimates of between 46.6 and 97.7%.
When we assessed specificity by HIV status, city, age, and sex, we found that it tended to be lower in HIV-positive individuals than in HIV-negative individuals. Specificity tended to be lowest in Kisumu and highest in Ndola. We did not find any evidence of differences in specificity between different age groups and between men and women.
The HerpeSelect and DiaSorin tests had specificities of 97 and 100%, respectively, when used with sera from industrialized countries (2) but were much less specific in our study (71 and 58%, respectively). Hogrefe et al. also found that HerpeSelect had a lower specificity in certain populations in sub-Saharan Africa than in the United States (9). These authors found a specificity of 100% in South Africa and Zimbabwe. However, among HIV-negative women in Kenya and Uganda the specificities were 81 and 70%, respectively. The overall specificity in the study populations in Kenya and Uganda was 88% and increased to 96% after an HSV inhibition assay was included in the enzyme-linked immunosorbent assay (ELISA) described in the product insert.
One possible explanation for the lower specificity of certain serological tests for HSV-2 infection in our study compared to what has been found in industrialized countries could be differences in the type of study population. Assessment of serological tests in industrialized countries is usually done on patients attending sexually transmitted disease clinics (1, 3), whereas we tested serum samples from a population-based survey.
A second possible explanation could be cross-reactivity with HSV-1 infection. Ashley et al. found that among patients in the United States with culture-documented HSV-1 infection the specificity of the Kalon test was 100%, while HerpeSelect gave a specificity of 93%, slightly lower than what has been found previously (3). This result suggests that HSV-1 infection might interfere with the performance of certain tests. In our study, however, we could not compare groups of individuals who were HSV-1 positive and negative, since >90% of the subjects in the multicenter study were infected with HSV-1. There was also no clear difference between tests that used purified gG2 as antigen and tests that used recombinant gG2. The two tests with the worst specificity, the DiaSorin test and the Captia HSV-2 Type-Specific IgG EIA, both used recombinant gG2, whereas one of the better tests, the Biokit test, used purified gG2 antigen.
A third possible explanation could be that some tests detect HSV-2 seroconversions earlier than other tests. The former tests would then have a “lower” specificity in populations with a high incidence of HSV-2 infection. A recent study found that the median time to seroconversion in serum sets of patients with primary HSV-2 infection was 21 days for HerpeSelect, 120 days for the Kalon test, and 87 days for Western blotting (3). However, if this were the explanation for the differences in specificity that we found in our study, we should have found a lower specificity of the HerpeSelect in younger age groups compared to older age groups, and this was not the case.
The scope for using HSV-2 serology clinically to detect infected individuals in sub-Saharan Africa is limited. In light of the high prevalence rates of HSV-2 infection, a strategy of testing and suppressive treatment with acyclovir does not seem to be feasible as a control strategy for HSV-2 infection in the general population. For the time being, HSV-2 serology in sub-Saharan Africa is mainly used for research purposes to assess the magnitude of the problem and to identify risk factors for HSV-2 infection. The implication of using a test with a suboptimal specificity is that the prevalence of HSV-2 infection will be overestimated. This is illustrated in Table Table2,2, where the prevalence ranged between 52 and 79%. In studies looking at risk factors for HSV-2 infection, associations are attenuated by a low specificity, and one may overlook potentially important associations. In studies in which HSV-2 is a risk factor, low specificity leads to associations being underestimated and to the failure to adjust analyses of other risk factors sufficiently for any confounding effect of HSV-2 infection.
In conclusion, it cannot be assumed when a type-specific ELISA is being chosen to detect HSV-2 infection that tests that perform well in industrialized countries will perform as well in other populations. More studies are needed on the performance of different HSV-2 ELISAs in different populations and on the reasons for differences in test performance.
We are grateful to Rhoda Ashley Morrow, who performed the Western blot analyses and who provided valuable comments on an earlier version of the manuscript.
We thank GlaxoSmithKline (United Kingdom) for financial support.
The members of the Study Group on Heterogeneity of HIV Epidemics in African Cities have gracefully agreed that sera from this study could be used for the evaluation of the HSV-2 serological tests. Members of the Study Group include A. Buvé (coordinator), M. Laga, E. Van Dyck, W. Janssens, and L. Heyndricks (Institute of Tropical Medicine [Belgium]); S. Anagonou (Programme National de Lutte contre le SIDA [Benin]); M. Laourou (Institut National de Statistiques et d'Analyses Economiques [Benin]); L. Kanhonou (Centre de Recherche en Reproduction Humaine et en Démographie [Benin]); E. Akam and M. de Loenzien (Institut de Formation et de Recherche Démographiques [Cameroon]); S.-C. Abega (Université Catholique d'Afrique Centrale [Cameroon]); L. Zekeng (Programme de Lutte contre le SIDA [Cameroon]); J. Chege (The Population Council [Kenya]); V. Kimani, J. Olenja (University of Nairobi [Kenya]); M. Kahindo (National AIDS/STD Control Programme [Kenya]); F. Kaona, R. Musonda, T. Sukwa (Tropical Diseases Research Centre [Zambia]); N. Rutenberg (The Population Council [United States]); B. Auvert and E. Lagarde (INSERM U88 [France]); B. Ferry and N. Lydié (Centre Franais sur la Population et le Développement/Institut de Recherche pour le Développement [France]); R. Hayes, L. Morison, H. A. Weiss, and J. R. Glynn (London School of Hygiene and Tropical Medicine [United Kingdom]); N. J. Robinson (Glaxo Wellcome [United Kingdom], formerly INSERM U88); and M. Caraël (UNAIDS [Switzerland]).
The sensitivities and specificities of HSV-2 EIAs were calculated as follows. The sensitivity and specificity of each test were estimated by using a modification of the method of Hawkins et al. (8). Sera were sampled for Western blotting based on the concordance or discordance of the four best tests (i.e., the MAb EIA, the Gull Premier HSV Type-Specific IgG EIA, and the Kalon Biological and Biokit tests) rather than the result of a single reference test. The concordance status of these four tests therefore took the place of a reference test in the analysis. The resolver test was a Western blot (Table A1).
Table A2 shows the calculation of sensitivity and specificity for the Kalon Biological test as follows: sensitivity = p(index test positive given true positive) = 0.566/0.613 = 92.3% and specificity = p(index test negative given true negative) = 0.377/0.386 = 97.6%.
Similar estimates were obtained if the MAb EIA was treated as the “reference test,” and Table A2 is replaced by a 2-by-2-by-2-by-2 table of estimated probabilities corresponding to the results of the index, reference, and resolver tests, as well as the concordance of the four best tests.
Confidence intervals for the sensitivity and specificity were calculated as described in the appendix to Hawkins et al. (8).