Do we agree on health state outcomes? A review of measurement agreement between time-trade off and other utility measures

Abstract

Aim: The aim of this review is to investigate the measurement agreement of time trade-off (TTO) and direct/indirect health utility measurements methods. Discrepancies have been reported between utility elicitation methods, thus the study objective was to collect all empirical studies that investigated measurement by Bland–Altman analysis (BA) and estimate overall means differences. Methods: A systematic literature review was performed in 2025 April, on three online databases (PubMed, Web of Science, Cochrane) following PRISMA guideline to synthetize (1) original, (2) English language studies, (3) investigating measurement agreement between TTO and other direct and indirect utility measures (4) by BA. Bayesian meta-analysis was performed to estimate overall mean difference and heterogeneity between measures. Results: Overall, n = 402 records were found, n = 41 assessed in full text and finally n = 12 studies were included into the synthesis. The studies covered nine different diseases, the mean TTO utility scores ranged between 0.96 (patient experienced myopia) and 0.42 (patient experienced colorectal cancer). The pooled means differences between the TTO and direct/indirect measures was small (-0.01 and 0.01), however the 95% lower–upper confidence intervals warns that mean estimates can deviate by 0.1 to 0.2. Moderate study heterogeneity (τ = 0.04 and τ = 0.13) also points on considerably varying utility results study-to-study. Conclusion: Between TTO and other direct/indirect utility measures our review found small mean differences, however significant between-study heterogeneity is indicating inconsistent measurement agreement. Currently, whether discrepancies arise from valuation technique, instrument properties, or study context remained undiscovered.