Motivated by the goal of evaluating a biomarker for acute kidney injury, we consider the problem of assessing operating characteristics for a new biomarker when a true gold standard for disease status is unavailable. In this case, the biomarker is typically compared to another imperfect reference test, and this comparison is used to estimate the performance of the new biomarker. However, errors made by the reference test can bias assessment of the new test. Analysis methods like latent class analysis have been proposed to address this issue, generally employing some strong and unverifiable assumptions regarding the relationship between the new biomarker and the reference test. We investigate the conditional independence assumption that is present in many such approaches and show that for a given set of observed data, conditional independence is only possible for a restricted range of disease prevalence values. We explore the information content of the comparison between the new biomarker and the reference test, and give bounds for the true sensitivity and specificity of the new test when operating characteristics for the reference test are known. We demonstrate that in some cases these bounds may be tight enough to provide useful information, but in other cases these bounds may be quite wide.

Diagnosis of bovine respiratory disease (BRD) in beef cattle placed in feedlots is typically based on clinical illness (CI) detected by pen-checkers. Unfortunately, the accuracy of this diagnostic approach (namely, sensitivity [Se] and specificity [Sp]) remains poorly understood, in part due to the absence of a reference test for ante-mortem diagnosis of BRD. Our objective was to pool available estimates of CI\'s diagnostic accuracy for BRD diagnosis in feedlot beef cattle while adjusting for the inaccuracy in the reference test. The presence of lung lesions (LU) at slaughter was used as the reference test. A systematic review of the literature was conducted to identify research articles comparing CI detected by pen-checkers during the feeding period to LU at slaughter. A hierarchical Bayesian latent-class meta-analysis was used to model test accuracy. This approach accounted for imperfections of both tests as well as the within and between study variability in the accuracy of CI. Furthermore, it also predicted the SeCI and SpCI for future studies. Conditional independence between CI and LU was assumed, as these two tests are not based on similar biological principles. Seven studies were included in the meta-analysis. Estimated pooled SeCI and SpCI were 0.27 (95% Bayesian credible interval: 0.12-0.65) and 0.92 (0.72-0.98), respectively, whereas estimated pooled SeLU and SpLU were 0.91 (0.82-0.99) and 0.67 (0.64-0.79). Predicted SeCI and SpCI for future studies were 0.27 (0.01-0.96) and 0.92 (0.14-1.00), respectively. The wide credible intervals around predicted SeCI and SpCI estimates indicated considerable heterogeneity among studies, which suggests that pooled SeCI and SpCI are not generalizable to individual studies. In conclusion, CI appeared to have poor Se but high Sp for BRD diagnosis in feedlots. Furthermore, considerable heterogeneity among studies highlighted an urgent need to standardize BRD diagnosis in feedlots.

Composite reference standards (CRSs) have been advocated in diagnostic accuracy studies in the absence of a perfect reference standard. The rationale is that combining results of multiple imperfect tests leads to a more accurate reference than any one test in isolation. Focusing on a CRS that classifies subjects as disease positive if at least one component test is positive, we derive algebraic expressions for sensitivity and specificity of this CRS, sensitivity and specificity of a new (index) test compared with this CRS, as well as the CRS-based prevalence. We use as a motivating example the problem of evaluating a new test for Chlamydia trachomatis, an asymptomatic disease for which no gold-standard test exists. As the number of component tests increases, sensitivity of this CRS increases at the expense specificity, unless all tests have perfect specificity. Therefore, such a CRS can lead to significantly biased accuracy estimates of the index test. The bias depends on disease prevalence and accuracy of the CRS. Further, conditional dependence between the CRS and index test can lead to over-estimation of index test accuracy estimates. This commonly-used CRS combines results from multiple imperfect tests in a way that ignores information and therefore is not guaranteed to improve over a single imperfect reference unless each component test has perfect specificity, and the CRS is conditionally independent of the index test. When these conditions are not met, as in the case of C. trachomatis testing, more realistic statistical models should be researched instead of relying on such CRSs.

There is conflicting evidence about the accuracy of bone scintigraphy (BS) for the diagnosis of complex regional pain syndrome 1 (CRPS 1). In a meta-analysis of diagnostic studies, the evaluation of test accuracy is impeded by the use of different imperfect reference tests. The aim of our study is to summarize sensitivity and specificity of BS for CRPS 1 and to identify factors to explain heterogeneity. We use a hierarchical Bayesian approach to model test accuracy and threshold, and we present different models accounting for the imperfect nature of the reference tests, and assuming conditional dependence between BS and the reference test results. Further, we include disease duration as explanatory variable in the model. The models are compared using summary ROC curves and the deviance information criterion (DIC). Our results show that those models which account for different imperfect reference tests with conditional dependence and inclusion of the covariate are the ones with the smallest DIC. The sensitivity of BS was 0.87 (95% credible interval 0.73-0.97) and the overall specificity was 0.87 (0.73-0.95) in the model with the smallest DIC, in which missing values of the covariate are imputed within the Bayesian framework. The estimated effect of duration of symptoms on the threshold parameter was 0.17 (-0.25 to 0.57). We demonstrate that the Bayesian models presented in this paper are useful to address typical problems occurring in meta-analysis of diagnostic studies, including conditional dependence between index test and reference test, as well as missing values in the study-specific covariates.

暂无摘要。

Studies that evaluate the accuracy of binary classification tools are needed. Such studies provide 2 × 2 cross-classifications of test outcomes and the categories according to an unquestionable reference (or gold standard). However, sometimes a suboptimal reliability reference is employed. Several methods have been proposed to deal with studies where the observations are cross-classified with an imperfect reference. These methods require that the status of the reference, as a gold standard or as an imperfect reference, is known. In this paper a procedure for determining whether it is appropriate to maintain the assumption that the reference is a gold standard or an imperfect reference, is proposed. This procedure fits two nested multinomial tree models, and assesses and compares their absolute and incremental fit. Its implementation requires the availability of the results of several independent studies. These should be carried out using similar designs to provide frequencies of cross-classification between a test and the reference under investigation. The procedure is applied in two examples with real data.