Obstetrical outcome valuations by patients, professionals, and laypersons: differences within and between groups using three valuation methods

Vignettes

First, we performed in-depth interviews with ten patients with GH, PE and/or IUGR and with ten obstetrical care professionals about the physical, psychological, and social burden and other consequences of GH, PE and IUGR. From these qualitative interviews, a list of 42 aspects emerged, which was aggregated into five attributes [24]: 'maternal health ante partum', 'time between diagnosis and delivery', 'process of delivery', 'maternal outcome', and 'neonatal outcome'. The attribute levels were chosen according to interviewees' responses, literature review, and primary and secondary outcome measures from the HYPITAT (ISRCT08132825) [21, 22] and DIGITAT (ISRCT10363217) [20, 25] trials. Each attribute had 2 to 7 levels; all were defined to be present with certainty (i.e. no risks involved).

We converted the attributes and levels into vignettes containing both a visual and a written representation (Additional file 1). The visual part depicts a time line to visualize the course of maternal and neonatal health over time. The time lines start when GH, PE or IUGR is diagnosed and they end one year post partum. A text box over the maternal timeline depicts the process of delivery: induction of labour, onset of delivery, and mode of delivery. Colours were used to display severity of health states; explanation of the colours and obstetric/perinatal terms were given on a detailed reference sheet (Additional file 2). Details of this procedure are explained elsewhere [24].

Design

The total number of usable unique vignette pairs was 37,990. Because of this large number of usable vignette pairs we applied an incomplete factorial design of 240 single vignettes for the VAS and TTO, and 120 paired vignettes for the DCE method (for details, we refer to [24]). We checked for the assumptions of orthogonality and level balance.

The 240 (120 paired) vignettes were distributed over six booklets. Each booklet consisted of two parts: 20 panel session vignettes (18 single VAS/TTO vignettes (9 paired DCE vignettes), plus the best and worst possible vignettes which we used for anchoring, and 26 home-assignment vignettes (22 single vignettes (11 paired vignettes), plus 4 single re-test vignettes (2 paired vignettes)). In this study we compared the outcomes between groups using just one of the six booklets. The other five were used for the larger study in which the outcomes of the total design was the objective [24]. All participants of current study valued the same set of 46 vignettes.

Participants

Participants in the valuation study were 24 patients, 30 obstetrical care professionals and 27 laypersons. The group of patients consisted of women who had a pregnancy that was complicated either by GH, PE, or IUGR and have participated to either the DIGITAT or the HYPITAT trials [22, 26]. These women participated in the study within six months after childbirth. The group of obstetrical care professionals consisted of gynaecologists, midwives, and residents in gynaecology, none of them with specific expertise in health state valuation, but all involved in the Dutch Obstetric Consortium (for more information, see http://www.studies-obsgyn.nl). They were recruited by email invitation. The laypersons were men and women over 18 years of age who had previously participated in valuation studies [27]. The laypersons and patients received a €50 participation fee.

Valuation methods

Each participant valued the single vignettes with a VAS and TTO, and each paired vignette with a DCE.

The VAS is a psychometric rating method with equal-interval categories [28]. Our VAS depicted a 100-point vertical thermometer ranging from 0: 'worst imaginable health state' (lower anchor) to 100: 'best imaginable health state' (upper anchor), the standard EuroQoL-format [29]. Each respondent was asked to draw a horizontal line on the VAS to indicate where the combined maternal and neonatal health state vignette should be positioned, taking the top and bottom anchors into consideration.

The aim of the TTO method is to elicit the maximum amount of time in full health that respondents are willing to trade to avoid a suboptimal health state [30]. Our TTO method involved a two-step procedure: first, the respondent had to state how much maternal time he/she was roughly willing to give in and, second, given the rough indication, how much maternal time he/she was exactly willing to trade (see Additional file 3) [31]. We specifically asked respondents to state how much time of the mother's life in full health he/she was maximally willing to trade off in order to attain full health for both mother and infant, given their health states as presented in the vignette. Respondents could trade-off between 0 days and 10 years of the mother's life.

The aim of DCE is to derive patients' preferences for a number of different aspects ('attributes') of a health state by presenting hypothetical choices between two or more scenarios in which the levels of the attributes are systematically varied [32]. In our study, respondents were invited to choose the best one of two alternative vignettes (forced choice) within a vignette pair. For an example of one vignette, see Additional files 1 and 2.

Study procedures

The study consisted of group sessions with 6 to 16 participants per group. There were two sessions with laypersons, two sessions with patients, and three sessions with professionals. The participants within each session were of the same respondent group. Each session was conducted by a trained moderator (DB, GJB, JAH, MFJ) who followed a detailed protocol adapted from the Dutch Disability Weights [33], MiDAS [27] and IBIS [34] protocols. Ethical consideration was not deemed necessary for this type of study.

In the group sessions, participants were invited to value the first 20 vignettes (18 single vignettes (9 paired vignettes for DCE), plus the best and worst possible vignettes) with first DCE, then VAS, and finally TTO. We explained the vignettes thoroughly, and respondents could practice on some sample vignettes in order to get used to the layout vignettes, the meanings of the used colours, and the weighting of the health states. The DCE task took about 10 minutes, the VAS task 15 minutes and the TTO task about 20 minutes. After the valuation tasks, participants filled out a questionnaire on background characteristics and their obstetric history.

Each session was followed by an individual home assignment. In the individual home assignment, the participants valued the remaining 26 vignettes: 22 single vignettes (11 paired vignettes for DCE), 4 single retest vignettes for VAS and TTO and 5 paired retest vignettes for DCE. They valued the vignettes in the same order as in the group session (first DCE, then VAS, and finally TTO). Finally, they completed a questionnaire on the user-feasibility of the written and visual components of the vignettes (response mode: 'comprehensible', 'neutral', 'incomprehensible'); the reference handout; comprehensibility of the five individual attributes; difficulty of each valuation method (response mode: 'easy', 'not easy but not difficult', 'difficult'); and the self-reported amount time needed to complete the home assignment. A telephone number was provided which the participants could dial if they needed assistance with the tasks or the questionnaires.

Analysis

We measured feasibility, reliability, and comparability of each combination of group and valuation method.

Regarding feasibility, differences between groups of the time needed to complete the home assignment was calculated using one-way analysis of variance (ANOVA) followed by Tukey's post hoc test. Linear regression analysis was used to determine the impact of sex, age, educational level, and respondent group on the time needed to complete the home assignment. Feasibility ratings between groups were compared with the χ2 test or Fisher's Exact Test.

Reliability was investigated by generalizability theory (G-theory) with restricted maximum likelihood estimation (REML) was used to determine the variation explained by respondent group in the VAS and TTO valuations. The test-retest reliabilities of the TTO and VAS were analyzed per group using intra-class correlation coefficient (ICC; two-way random effects, single measures, absolute agreement; 95% CI). The DCE test-retest reliability was assessed using Cohen's unweighted kappa (κ). Within-group consistencies of the VAS and TTO were calculated per group using ICC (two-way random effects, single measures, absolute agreement; 95% CI) to measure the rate of consensus within each group.

The DCE score for a health state was indirectly derived by adding up all attribute level coefficients of the health state (βXi is the coefficient of attribute X, level i). The mean transformed (VAS and TTO) and the indirectly derived (DCE) vignette scores were calculated for all 46 presented vignettes. The correlation between each two valuation methods per respondent group was plotted to visualize group clustering in valuations and to expose valuation tendencies per group-method combination.

The relative attribute weights (coefficients) were calculated per group for the VAS and TTO using linear regression of the transformed VAS and TTO scores, and by application of multinomial logit (conjoint analysis) on the DCE scores. ICCs were interpreted according to the guidelines of Landis and Koch [36]. The estimated relative attribute weights were compared between groups within methods, and between methods within groups, with Kendall's Tau-b correlation coefficient.

Analyses were conducted using SPSS 15.0 for Windows (SPSS Inc). Multinomial logit was performed using SAS 9.1.2 (SAS Institute Inc). A p-value < 0.05 (two sided) was considered to indicate statistical significance.

Baseline characteristics

User-feasibility

The response rate to the home assignment was 96% for the patients, 73% for the obstetrical care professionals (residents: n = 12, 86%; gynaecologists: n = 7, 64%; midwives: n = 3, 60%) and 89% for the laypersons.

The mean amount of time needed to complete the home assignment was 45 minutes for the patients, 49 minutes for the obstetrical care professionals, and 75 minutes for the laypersons (laypersons versus patients p = 0.001; laypersons versus obstetrical care professionals, p = 0.007). Respondent's age had a significant impact on the amount of time (in minutes) needed to complete the home assignment (β = 0.315; p = 0.034, for higher age) while sex and education did not (females: β = 0.065; p = 0.598; > 11 years of education: β = 0.182; p = 0.168).

Comprehensibility of the vignettes was equal across groups (p = 0.861) and the visual and the written components of the vignettes were equally comprehensible across groups (p = 0.549). The DCE was overall rated as 'easy', the VAS as 'not easy but not difficult', and the TTO as 'difficult' (no differences between groups; p = 0.611, p = 0.746 and p = 0.738, respectively).

Reliability

The overall within-group reliability coefficients indicated substantial agreement on the VAS; all groups displayed substantial within-group reliability. Overall within-group reliability was low for the TTO, with large differences between obstetrical care professionals versus patients and laypersons. The DCE had overall substantial within-group reliability, with some variation between groups.

Comparability

Figures 1A, B and 1C display the valuations of each group, for each pair of valuation methods respectively. The association between the transformed VAS and TTO scores per group is shown in Figure 1A. Participants are apparently less willing to trade off maternal life time for health states with VAS score > 0.50, a pattern consistent across respondent groups. For health states with VAS score < 0.50, all groups are willing to trade off maternal life time, but to a various degree; patients are generally willing to trade off more time than laypersons. The transformation of the TTO scores using formula (2) did not result in the intended linearity of the scores.

Figure 1B depicts the association between VAS and DCE scores per group. Over the entire VAS spectrum, patients and laypersons express systematically higher preference scores (in terms of DCE) than the obstetrical care professionals. The linear regression lines of the patients and obstetrical care professionals are parallel, with patients displaying a systematic higher valuation. The scores of laypersons are more in agreement with those of the patients at the lower end of the VAS, but more in agreement with the obstetrical care professionals at the upper end of the VAS spectrum (VAS > 0.75).

The scale variance heterogeneity was tested for VAS using the anchoring (both best and worst) health state vignettes. The worst and best possible vignettes resulted respectively in mean VAS scores of 0.13 and 0.96 for lay people, 0.05 and 0.92 for patients, and 0.11 and 0.91 for professionals. Using ANOVA with Tukey's post hoc test, the worst vignette differed between lay people and patients (p = 0.029) and the best vignette differed between lay people and patients (p = 0.026) and between lay people and professionals (p = 0.007).

Figure 1C shows the relationship between the transformed TTO and DCE valuations. For DCE scores > 0.90, respondents are less willing to trade off maternal time on the TTO; this pattern is present in all groups. For DCE scores < 0.90, all groups are willing to trade off maternal life time. Patients are again relatively more willing to trade off maternal life time than laypersons and professionals. From these figures it is apparent that the threshold score to trade off depends on the method chosen.

Table 4 presents the relative attribute weights (coefficients) for each valuation method per respondent group. Increasing relative weights with decreasing optimality of the health states is overall most consistent in the VAS. The TTO suggests a skewed relationship between health state and relative weight; while mild and moderate health states have low relative weights, severe health states (neonatal outcome) have very high relative weights; an observation that is consistent in all groups. The DCE shows another pattern: relative weights increase with the severity of the health states (neonatal outcome), but less strongly than VAS and TTO. Figure 2 shows the aggregated relative weights (i.e. mean relative weight over the methods) per respondent group, per attribute category: process, maternal outcome, and neonatal outcome.

Attribute and level (baseline)	VAS			TTO			DCE
	Patients	Professionals	Laypersons	Patients	Professionals	Laypersons	Patients	Professionals	Laypersons
Maternal health ante partum (normal)
- Moderate	-0.020	-0.010	-0.006	0.025	0.023	0.003	0.003	0.013	0.031
Time between diagnosis and delivery (3 days)
- 1 week	-0.024**	-0.016*	-0.003	0.005	-0.003	0.017	0.053*	0.050	0.054*
- 2 weeks	-0.013	-0.022*	-0.014	-0.003	-0.020	-0.022	-0.042	0.001	-0.057*
Process of delivery (Cervical)
- Induction, cervical	-0.081**	-0.069**	0.045*	0.028	0.044	0.080	0.023	-0.110	0.176
- Vacuum	-0.023*	-0.005	0.004	0.009	0.011	0.019	0.039	-0.065	-0.058
- Induction, vacuum	-0.023	0.008	-0.006	0.025	0.035	0.030	0.113	-0.065	0.008
- Cesarean section (planned)	-0.045**	-0.005*	-0.047*	0.025	0.021	0.008	0.093	-0.131	-0.066
- Cesarean section (not planned)	-0.042**	-0.069*	-0.047*	-0.019	0.006	-0.006	0.093	-0.076	-0.081
- Induction, Cesarean section (not planned)	-0.042**	-0.045	-0.047*	-0.030	-0.030	-0.047	0.025	-0.078	-0.089
Maternal outcome (3 days moderate)
- 3 days severe and 4 days moderate	-0.033**	-0.034**	-0.030*	-0.010	-0.008	0.014	-0.072	-0.021	0.053
- 7 days severe and after 1 year moderate	-0.070**	-0.146**	-0.113*	-0.024	-0.066	-0.052	-0.126*	-0.116	-0.005
Neonatal outcome (No complications)
- 7 days moderate	-0.042**	-0.031**	-0.022*	0.008	-0.003	-0.003	0.076**	-0.016	0.079**
- 3 days severe and 7 days moderate	-0.087**	-0.077**	-0.072**	0.001	-0.015	0.003	0.010	-0.001	0.061
- 10 days severe and after 1 year moderate	-0.169**	-0.154**	-0.179**	-0.099**	-0.101**	-0.074	-0.093**	-0.087	-0.108**
- 3 days severe and death	-0.287**	-0.309**	-0.366**	-0.688**	-0.615**	-0.623**	-0.138**	-0.172*	-0.074**

* p ≤ .05 ** p ≤ .01

From the group perspective, correlations between relative weights obtained with each pair of valuation methods were consistently low to moderate across groups of respondents. VAS-TTO correlation was highest among patients and professionals (τ = 0.37). VAS-DCE was highest among patients and laypersons (τ = 0.28), while TTO-DCE correlation was highest among professionals (τ = 0.44). When comparing each two valuation methods across groups, the laypersons give the most inconsistent valuations between methods.