Pre-pilot assessment

Between 1 and 19 July 2013, a small formative research study was done in the target area of Kampong Tralach OD in Kampong Chhnang Province. This consisted of three pre-pilot assessments. Firstly, two brief interviews with local health authorities at the OD and Provincial Levels regarding the status of recent neonatal mortality. Secondly, we undertook a pretested quantitative questionnaire on 78 randomly selected mothers with newborns to ask about their attitudes regarding of piloting an mHealth program. Thirdly, midwives in the target health administrative area were also interviewed using another pretested questionnaire at the health centre and referral hospital level regarding their perspectives of a mHealth intervention in this area. The results of the formative study fed into the design of the mHealth pilot.

Design and description of the ‘baby village care’ program

The formative research results were incorporated into the design of the program. The ‘Baby Village Care’ program primarily targets mothers with newborns aged under 28 days who have access to a mobile phone. Our platform, an online cloud program entitled ‘Verboice’, used IVR technology to send seven voice-messages 60 to 90 s long directly to the mothers’ mobile phone starting from Day 3 of the newborn’s life. Verboice would call to the registered mobile phone number every 4 days until the newborn is 28 days old. Each message uses the different voices of several respected village ‘characters’ such as a doctor, a midwife, a village health worker, another mother, the village chief and an elderly grandmother. Given the novelty of mHealth programs, trained midwives at the health facilities were chosen to be the ‘gatekeepers’ of access to the service. Midwives from the target area received a half day interactive, hands-on training on how to register eligible mothers before the pilot was launched. Registration onto the service is done after the participating mother has given birth and before they are discharged from the health facility. We designed the pilot intervention with sustainability in mind and a conceptual framework was used to project future plans for the program that involved a model of self-funding from Public Private Partnerships. This is the unique feature of the pilot as often these are implement without a long-term goal in mind [12]. (Suggested placement of Fig. 1: Conceptual Framework).

Pilot intervention and evaluation of ‘baby village care’

The evaluation of the pilot program took place from between 16 and 25 December 2013 with six trained enumerators, two of which also conducted and facilitated two Focus Group Discussions (FGDs). In the pilot evaluation, we interviewed 128 of the 455 participants using a pretested and back-translated quantitative questionnaire. These participants were randomly selected from the registered list of phone numbers recorded on the Verboice program and were interviewed over the phone. Two FGDs were conducted with one facilitator and one scribe. One FGD consisted of three mothers and three community health workers, the other consisted of all health workers (midwives and community health workers). The quantitative survey data was entered into SPSS version 20 and the results of the pilot evaluation were performed using SPSS version 22. Frequency analyses were mostly used to yield the results reported here.

Acceptability

A key aim of the pilot intervention was to assess whether or not this type of intervention would be acceptable to the target audience and thus feasible for potential scale-up. The results indicate that acceptability was high with all mothers listening to at least four of the seven messages. Sixty-one percent mothers surveyed report that they told others about the service, 71% would recommend the service to other mothers and 63% reported sharing the information they heard with others. The results from our FGD with the health workers also responded well to the program. They report that the program was a good way of connecting with mothers, as one midwife said ‘This (initiative) makes mothers more aware of how to take care of (their) babies’. Our quantitative results indicate that 85% of the mothers surveyed listened to some messages not just once but several times, using the option in the IVR to repeat again, allowing the messages to be heard by other members of the household such as their husbands (n = 87), mothers (n = 67), siblings and older children (n = 23). The FGD results also showed that some first-time mothers said they had ‘shared the messages for other members of the household to listen to’.

Further, 96% of respondents reported the length of the messages (60 to 90 s) was appropriate and all respondents reported that the intervention was helpful for them when looking after their newborn. Additionally, 45% also reported learning something new from listening to the messages. The majority of respondents (96%) report that they would like to see the service extended, 44% of which suggested a service extension of at least 12 months. Finally, results also showed that 43% of mothers reported that they took their babies to the health centre as a result of listening to the messages.

Technology

We also assessed a number of technological points to judge user satisfaction. The timing of the automated calls (between 1800 and 2000 h, as suggested during the pre-pilot formative research) was one aspect of importance because we wanted to target a time when the mother is at home to receive the message and increase the chances of the messages being heard. Our evaluation results indicate that 36% of respondents received a call during the preferred time slot and the majority received calls at other times of day. Despite this, only 6 of those surveyed reported that they thought the timing of calls was inconvenient. The main reasons for inconvenient call time were that they did not have their phone with them at the time of the call or they were too busy to answer the call. The technical aspects of line rental provided by the private telecommunications company was also assessed (i.e. call clarity, sudden cut-offs). All respondents surveyed reported the line was clear and only 12 out of those surveyed reported their call was cut off while listening to the messages. Discussion during the FGDs included one first-time mother mentioning that ‘At anytime (when I was listening to the message) the line was clear. The timing was convenient for me because normally mothers with new babies stay home after birth and do not go anywhere’.

Willingness to pay

To make this type of mHealth intervention financially sustainable willingness to pay was assessed. A majority (60%, n = 75) of respondents said that they would be willing to pay for the service. The fee ranged being between 100 and 3000 Cambodian Riels (4000 Riels = 1 USD) with an average willingness to pay of 577 Riels. One mother from the FDG remarked ‘I would like to pay USD 0.10 for the messages because they are very important to child health’. Despite this willingness to pay reported from the mothers, it should be noted that during the FDG with the midwives and community health workers, they reported that the program should be made free to mothers as they suspected that ‘most mothers do not have enough credit on their phones (to pay for the messages)’.

IVR versus SMS

In this pilot study, we established that a mHealth intervention using IVR technology is able to engage mothers and families in neonatal health care in Cambodia. Despite the popularity of using SMS as part of a mHealth program, particularly in Africa as reminder services [13, 14], we chose to use the IVR platform to send voice-messages to empower mothers with health knowledge. This is due to the fact that a recent local report showed that only 29% of the mobile phones in Cambodia support the local Khmer script [8]. Couple this with low levels of literacy among rural women, voice-messaging was the better option for this intervention. A similar program by Crawford et al. in Malawi, indicated that there was no difference in the effectiveness between either IVR or SMS modalities [15]. It reports that both methods can be highly effective and supported by the target users [15].

Contextual feasibility

One of the criticisms of mHealth projects is that they lacked contextual framework [6, 12]. Contextualising the program into the intervention’s design ensures a greater chance of program success [16, 17]. We implemented our pilot within our conceptual framework (see Fig. 1). The pilot was the first piece of a bigger puzzle that part of a bigger plan to build a sustainable program based on cultural, financial, and technological solutions. Therefore, the results of the pilot evaluation will be used as a base for the next phase (currently underway) which will include buy-in from high level government stakeholders as well as private partners (for example mobile telecommunication providers). This kind of enabling environment from the get-go can be the foundation on which the interventions like this can be scaled up in a step-by-step approach.

Cultural feasibility

The cultural aspect of our intervention was the content of our voice-messages. The dramatic ‘role play’ style meant that listeners trusted the messages which were inspired by the successful Aponjon Program in Bangladesh [18]. There, a similar type of dramatic style was used to build rapport with listeners, facilitating a supportive environment for the intervention’s acceptance [18, 19]. It has been criticized that mHealth initiatives in developing country contexts are more often than not, developed in English rather than the local language [12]. However, in our case, we developed the text in both English and Khmer languages simultaneously. The Khmer version then field-tested to ensure the correct cultural meaning was relayed and any nuances were well understood. The outcome that mothers were happy to recommend the service to other mothers is a key demonstration of trust in the program and the information it provided. Additionally, other household members also listened to the messages due to a repeat function in the IVR pathway. Results show that these ‘other listeners’ were often the husband, mother and sibling of the primary listener. In Cambodian culture, the primary decision holder is the head of the household, usually the husband. However, history has suggested that the Khmer culture follows a matriarchal system [20], hence the maternal grandmother (who often lives with the family) also has great decision-making power, particularly with regards to child rearing. This is likely to have a significant influence on the empowerment of the mother, especially on issues of baby care if all three (the targeted mother, her husband and her mother) are in agreement to the same information and action. Matsuoka et al., showed that in their qualitative research, the beliefs and opinions of the Cambodian family elders played a significant psycho-cultural role in the decision making power of the mother to seek professional medical care [21]. While women and mothers are often the target of mHealth interventions, particularly around maternal and child health, it is also crucial for programs to offer the chance to communicate with men and the youth to support a family style approach to see real behaviour change. Our method to include both male and female respectable characters into the messages is likely to have had an impact on the believability of the messages. This correlates with our results that indicate the intervention was able to prompt mothers to take the baby to the health facility as a result of listening to the messages.

Technological feasibility

While there are high phone ownership rates in Cambodia, there is also the issue of owning more than one SIM card. In fact, more often than not, ownership of up to four SIM cards, all on different networks has been reported [8]. This is great challenge when the mHealth intervention is only registered on one SIM. There are four major telecommunication providers in Cambodia and SIM card switching is a frequent occurrence and a hindrance to interventions such as ours where only one number from one network is registered [9]. While our feasibility study reports that telecommunication lines were clear with only some calls being missed, having more than one SIM is one of the biggest barriers to the end-user’s ability to access all the messages in a timely manner. One of the limitations of our evaluation was that we did not ask whether or not the number used for registration was the ‘primary’ number being used and if indeed our target audience switched SIM cards.

Economic feasibility

One major limitation to a mHealth program’s sustainability is its funding [3, 6]. Some types funding arrangements (namely donor funding) can predispose the systems to sustainability challenges [6]. While a large majority of the mHealth pilots and scale-up interventions are being implemented using donor funding [14], we wanted to create a program that would eventually fund itself. Though our initial pilot was free, the fact that our participants want the program extended and were willing to pay a small fee for it, suggest that they see the value in the content of the messages. This echoes the findings of the Aponjon program’s formative research report where 76% and 66% of pregnant women and new mothers respectively said they were willing to pay a ‘small fee’ for the service [22]. However, the amount users were willing to pay was much higher in our context (0.01 USD versus 0.13 USD). Unlike traditional preventative and curative health services, the addition of fees can abruptly decrease the utilization of the service or quality of service [23, 24]. Our results indicate that the willingness to pay has the potential to engage further utilization without compromising on the quality. This is because the quality of the content of voice-message is controlled by the implementing NGO, not a third party. mHealth is a very new concept to rural Cambodian mothers and to our knowledge, this was the first intervention of its kind in Cambodia. For there to be a willingness to pay for a service like this could mean a great potential to scale-up and may see real health impacts if the enabling environment is supported (socially, economically, technologically). However, the downside to a fee-based approach would limit the program to those who are willing and able to pay, thus neglecting the poorest of the poor who may be unable to do so. We envisage that the program would be offered for free to those who have an ID Poor card (a government concession card given to those who meet a set of poverty criteria). To pay for these users, based on our conceptual framework, we expect a public private partnership could be an ideal way for any fee off-sets and scale up in the future. The intention of a user-paid service was to not devalue the messages. The idea of offering the program free for everyone may be more equitable, but if people were willing to pay for something, it honours their higher commitment and value to what is being purchased.

Strengths

A strength of this feasibility study is its bottom-up approach guided by a conceptual framework. It uses a family theme to be inclusive of not just mothers but their husbands, family elders and siblings. The pilot also considered social-political norms through early engagement with local authorities and using the existing health infrastructure as well as considering cultural and environmental influences.

Limitations

The intervention period of four months is short and the exposure time to the intervention was four weeks. The results are likely to indicate some recall bias depending on how well mothers remember the intervention compared to when they were surveyed for the pilot evaluation. For example, some respondents recall receiving ten messages. Given there are only seven messages in total, this could be deduced to recall error or a technical issue where some messages were sent more than once (neither was investigated for in this instance). Finally, we did not collect any information on the literacy, education level or social economic factors of the targeted mothers. Such factors may have influenced the outcome of the pilot intervention. Our study focused on the feasibility of a mHealth intervention of this kind, however, we did not have the resources to assess the effects of the intervention on health outcomes. Our messages were also only available to mothers who delivered their children in a health facility, which meant that those who still delivered at home who maybe more likely to be a poorer, more vulnerable group are still at risk.

Recommendations for similar mHealth or mobile technology interventions in other settings

For others considering implementing a similar project in other settings, we recommend that detailed pre-pilot work be considered to assess the community and local authority buy-in to the mHealth idea if it is novel. The results of this can feed into the design of the program and can be of valuable importance. During the development of the program content, based on our experience and results, the more realistic the messages can be, the more likely the target audience will be to responding to them. In our case, we tried as much as possible to use ordinary local wording, spoke clearly and slowly, using ‘trusted’ voices such as a village chief, a midwife, a peer mother etc. Though our pilot targeted mothers with infants, a largely female audience, we did not limit our messages to women only, but also reached out to men as they are a key part of the decision making process. As such, for other programs wanting to use IVR technology, we would suggest considering messages (their content and ‘voice’) to be as broad-reaching as it is possible, as it was in our case, so that those other than the targeted mobile user may also be listening in. However, on the flip side, if the program being implemented is to discuss highly private content (such as issues of domestic violence), consideration will need to be given to the potential that there may be other secondary users of the mobile. Finally, depending on the technology platform being used, try to make the system as user-friendly as possible. Consider putting fail-safes in place wherever possible so that if a mistake were to be made, there would be an option redo for the user.