The worldwide shortage of providers to meet mental health (MH) needs calls for innovative solutions to expand access to care and improve patient outcomes. Depression is a leading cause of global disability; evidence shows that investment in screening and treatment yields substantial health and economic returns [1,2].

In underresourced communities, there are greater barriers to MH care, which calls for context-specific and creative solutions. One factor that may prevent people from seeking appropriate care is MH stigma, which has been found to be higher in racial and ethnic minority populations [3]. Minority groups, including Hispanic and Black communities, are disproportionately affected and less likely to receive treatment [4]. Those who do receive treatment often receive lower-quality services [5].

To address these specific barriers to MH care, there is a need for increasing the supply of providers that exercise cultural humility, as such approaches are more effective at addressing the MH needs of culturally heterogeneous populations [6]. Populations impacted by discrimination and racism often express mistrust of nonminority providers [7]. Addressing these MH care gaps includes expanding appropriate training in cultural competence and instilling humble approaches of care in existing providers [8]. Thus, innovative approaches to training additional providers are needed to meet the growing MH needs of underresourced communities.

MH task-sharing is a promising approach for increasing culturally sensitive MH services, particularly in underresourced communities [9]. Task-sharing shifts certain MH services from specialists to less-trained providers [10]. Studies show that task-sharing simple treatments, such as problem-solving therapy and medication management, have been implemented effectively [11,12].

Training and supervision are essential for successful task-sharing, as highlighted by the World Health Organization [13]. Systematic and locally specific efforts are needed to enable intervention sites to design and monitor their own training and supervision [14]. Additionally, simpler therapies, such as behavioral activation delivered by more junior MH workers, have been found to be as effective as more complex therapies delivered by specialists in improving outcomes for patients with depression [15]. Task-sharing interventions are also strengthened by incorporating the services of providers with strong community ties, shared lived experiences, credibility, and nonstigmatizing attitudes [10]. By training these community members and providers who are usually already providing informal support, communities can build their capacity to provide trustworthy MH services.

Resolving challenges associated with supervision and training is essential to improving MH treatment. Although clinical supervision and role-playing are useful training techniques, the conflicting priorities of task-sharing trainees often make it difficult to maintain regular training schedules, hindering the effective implementation of these methods [16]. Additionally, less experienced personnel may struggle to identify and treat subtle symptoms, such as reading facial expressions in patients with severe schizophrenia [17]. This underscores the importance of realistic simulation and accessible, flexible training tools. Traditionally, realistic clinical scenarios have been provided by standardized patients (SPs), trained persons who simulate actual patients and their symptoms [18]. However, this approach is resource-intensive, requiring significant time, travel, financial cost, and supervision [19,20].

Virtual standardized patients are computer-generated, interactive agents designed to simulate clinical presentations, allowing for repeated practice of MH skills such as interviewing and counseling [21]. Because virtual standardized patients do not require the same level of real-time oversight from supervisors, they can alleviate the burden on trainers by enabling self-paced, independent learning.

Virtual reality (VR) is described as a fully immersive virtual environment, while augmented reality (AR) overlays virtual elements onto the real world [22,23]. AR can foster a strong sense of presence and embodiment, making it a powerful tool in educational and training settings [24,25]. According to research, AR-driven identification and transportation can meaningfully affect users’ behaviors, physiological reactions, and psychological perception [26-31].

Social cognitive and situated learning theories support VR’s potential for observation-based learning in realistic contexts [32]. VR’s immersive and interactive nature also enhances experiential learning, making it effective in training by fostering empathy and user-centered thinking [33]. VR enables high-presence interactions with virtual patients (VPs) and creates realistic training environments [34].

Experiential learning through AR and VR in MH improves users’ comprehension, memory retention, and the development of practical skills by allowing them to participate in interactive simulations that closely mimic real-life scenarios [35]. These simulations elicit emotional responses similar to those experienced in real-life situations, nurturing empathy and profound emotional understanding, and providing valuable educational opportunities [36]. Additionally, AR and VR allow learners to practice in realistic environments, developing key skills like procedural recall, communication, problem-solving, and coping strategies [37]. Furthermore, AR and VR have been shown to be interactive, engaging, and convenient, hence making them valuable tools for virtual simulations in MH training and task-sharing settings [38].

VR training is increasingly used in health care education for both technical and nontechnical skill development [39,40]. In nursing education, a review of virtual simulations found them to be as effective as, or more effective than, conventional methods, offering self-paced, economical, and space-efficient learning opportunities [41].

In MH nursing, VR simulations facilitate emotional connections between learners and VPs by presenting realistic, interactive scenarios [42]. While earlier studies used nonimmersive, desktop-based VR, immersive VR with head-mounted displays (HMDs) has shown greater efficacy due to its sensory immersion, which intensifies focus, presence, emotional engagement, and learning outcomes [43-48]. In a usability test, a 360-degree VR simulation portraying schizophrenia scenarios was found to be user-friendly, engaging, educationally relevant, and highly immersive [49]. Another study on VR simulation training for behavioral health anticipatory guidance and motivational interviewing found that VR significantly increased pediatric residents’ engagement and critical skill development by providing an immersive, accessible, and realistic but patient-free learning and practice environment, and reduced their anxiety and stress [50].

AR is easier to use and has fewer negative physical side effects than VR, including the “cybersickness” that is sometimes felt after long VR usage [51]. AR also increases knowledge transfer by including real-world context that VR does not have [52]. In the medical field, AR is gaining popularity as a tool for clinical training and patient education by enabling medical professionals to visually communicate information about new treatments, medication mechanisms, and surgical procedures [53-56]. AR’s applications in anatomical and physiological training are noted for their maturity, showing substantial promise for transforming medical education through innovative prototypes and teaching approaches [57]. An integrative review by Zhu et al [58] found that AR has the potential to improve health care education by lowering failure rates and enhancing precision, while other studies also highlight AR’s role in enhancing knowledge retention, concept integration, and confidence among medical students [58-62]. For example, a pilot study contrasting traditional mannequin-based training with AR-enhanced approaches for patient decompensation recognition demonstrated that AR significantly improved participants’ assessment skills and clinical confidence [63]. Despite promising early findings, there are still relatively few published experimental and observational studies assessing AR’s application and efficacy in medical education [23].

AR in medical education has shown promise in enhancing learning and practical skills, and it has further potential in providing immersive environments for the development of important social skills [64]. By superimposing virtual components within the real environment, AR is very suitable for effective simulation training [58]. Ward et al [65] expressed that exposing users to challenging or rare scenarios in AR simulations creates a safe environment for practicing adaptive skills needed in the clinical space. Despite AR’s potential to deliver intricate and precise training across diverse social skills applications, the literature remains lacking, especially within MH training. A recent systematic review pointed out that while simulation-based training in health care education is advancing, there is limited systematic research on its impact on learning outcomes, with most studies to date focusing on feasibility and face validity [64].

Using an AR tool for MH simulation training with VPs is novel; hence, usability testing, defined as the evaluation of elements that impact users’ experience with a product for its intended purpose, is an essential first step to determine the strengths, weaknesses, and effectiveness of using AR for task-sharing [66]. Thus, we conducted a mixed-methods pilot study assessing the usability of our AR tool.

A convenience sample of participants was recruited from CIMH’s community network through listservs and word of mouth. Inclusion criteria were broad to capture a wide range of task-sharing perspectives, requiring that participants were members of or adjacent to the community served by the HSI and could speak to the needs of local MH training. Participants recruited for this study group did not overlap with study participants in the main HSI study. Education and MH training levels were varied to generalize the usability of the training tool; participants ranged from a college student with no MH training to an experienced licensed clinical social worker. While five participants were initially recruited and consented, one participant withdrew before beginning the user testing stage. Consequently, the final analytical sample consisted of four participants (n=4), representing a 20% proportion of missing data for the simulation and poststudy measures.

There were 5 back-to-back usability tests scheduled in a day, with each session lasting between 45 minutes and 60 minutes. Participants were compensated with a US $50 Amazon gift code for their contributions. Their informed consent was first obtained through a physical form, after which we carried out a five-stage data collection procedure:

Participants first engaged in a semistructured prestudy interview, detailing their training background, including their experience with CUNY’s MH skills modules and their role in the community. They also discussed the challenges and strategies associated with interactions with distressed community members who could be experiencing MH issues. (Multimedia Appendix 3).

Participants were then briefed about the study goals and AR simulation objectives. They were provided with clear instructions about the AR glasses usability test, emphasizing the experience over their knowledge.

After an orientation to the AR glasses, participants navigated an AR simulation, practicing the administration of the PHQ-4 form with a VP. To facilitate the “think-aloud” protocol, researchers provided a structured briefing and used standardized verbal probes during the simulation. Participants were prompted to describe visual elements (eg, “Describe what you see”), instructed on how to handle interaction cards (eg, “Show us how you would proceed”), and asked targeted questions during planned pauses regarding the character’s body language and the use of the PHQ-4 task. If participants deviated from expected interactions, researchers asked clarifying questions such as “What did you expect to happen?” to capture their cognitive processes.

A semistructured poststudy interview guide was used to assess participants’ perceptions of the usability of the AR simulation (Multimedia Appendix 3). Interview guide questions were adapted from usability testing literature [73-75]. The interview aimed to query feedback on the prototype’s use, effectiveness, and potential improvement areas. At the end of each interview, the researcher confirmed all relevant information had been included to ensure the quality and accuracy of the data.

Finally, a 17-question poststudy questionnaire (Multimedia Appendix 4), adapted from the PSSUQ with a 7-point Likert scale ranging from strongly agree (1) to strongly disagree (7) with N/A as an additional option [70], was administered. The PSSUQ is a validated instrument with high internal consistency (typically Cronbach α>0.89), designed to measure user satisfaction across system usefulness, information quality, and interface quality. Additionally, demographic questions with multiple choice options were included that determined Table 1.

The testing took place at one of CIMH’s facilities, in a large private room with several tables and chairs. The first and second authors conducted the usability tests and were the only people present in the space besides the study participant. The hardware used includes the Magic Leap 2 and two laptops for filling out the study questionnaires. The prototype was developed in Unity, using Ready Player Me for the custom avatar, Salsa LipSync Suite for facial animations, and Mixamo for body animations [76-79]. The usability lab featured two facing chairs (Figure 4), one for the participant and one for the VP. Researchers were positioned outside the participant’s field of view (Figure 5) and intervened only for technical issues or at specific checkpoints (Multimedia Appendix 3).

We used descriptive and summary statistics to analyze participant demographics and scores on the adapted PSSUQ. Due to the pilot nature of the study and the small sample size (n=4), inferential statistical testing was not performed; data are presented as mean (SDs) to indicate trends.

The PSSUQ consists of a set of 17 questions on a 7-point Likert-scale (1=strongly agree to 7=strongly disagree). The overall PSSUQ score is the average of the scores of questions 1 to 15 and 17. The subscales include: system usefulness (SYSUSE), which is the average score of questions 1 to 6, information quality (INFOQUAL), which is the average score of questions 7 to 12, and interface quality (INTERQUAL), which is the average score of questions 13 to 15.

All interviews were recorded using Zoom, with identifying details removed to maintain confidentiality [80]. Initial transcriptions, facilitated by Zoom’s automatic service, were refined by the first author, taking out any names, occupations, and other identifying details and using a prescribed participant number instead. After transcriptions were verified for accuracy, they were entered into the qualitative analysis software ATLAS.ti (ATLAS.ti Scientific Software Development GmbH), where a 2-step coding process was adopted [81].

Two independent coders (VN and SF) were used to minimize bias and positionality and ensure accurate depiction of participants’ perceptions [82]. The first author (VN) and a research assistant (SF) used an inductive approach based on conventional qualitative content analysis to analyze the interview data [83,84]. In contrast to summative content analysis, which focuses mostly on counting and measuring data, this method emphasizes the identification of recurrent patterns throughout the data collection, similar to traditional thematic analysis [85].

The researchers independently reviewed the interviews and subsequently coded key concepts in line with the interview questions while also drawing from codes from the participatory design phase, such as “Challenges,” “Workflow,” “Ease of use,” “Immersion,” “Usefulness,” and “Future recommendations.” Then, the researchers discussed preliminary findings since multiple analysts for code development enhance the findings’ credibility [82]. Using an open coding strategy, codes were refined and added, reflecting significant patterns in the data. Discrepancies in coding led to discussions until consensus was reached. The frameworks described in a related study by Hess et al [86] served as further guidance for the coding process. Domains from the study like “Experiential satisfaction,” “Learning engagement,” “Technology learning curve,” and “Opportunities for improvement” formed the basis for theme identification.

Transcripts were then restructured according to common codes for thorough examination of themes and patterns. Participants’ frequently overlapping and closely related codes were grouped together to create categories, which in turn led to the formation of primary themes. The categories were formed and summarized individually. Comparing these categories and summaries between the 2 researchers constantly and discussing to reach consensus allowed for principal themes (Multimedia Appendix 5) to be formed with a consistency of more than 90%, suggesting additional reviews were unnecessary [82].

Building trust with the community was highlighted as crucial by two participants. P1 told us that they

While P3 highlighted that

P3 further described:

Participants experienced difficulty adapting theoretical knowledge to practice. P2 noted that

While P1 emphasized that

P1 told us that in the current workflow they: “covered The Diagnostic and Statistical Manual of Mental Disorders,” which is a comprehensive classification system published by the American Psychiatric Association (APA), and “did a lot of role-play.” They added that “normally, when we do role-plays in class, nobody goes down that road of re-enacting more challenging scenarios like schizophrenia, people tend to present the lighter stuff that might not be entirely accurate.” This highlighted that current training included role-play and usually did not include difficult scenarios, which corroborated with our prototype design to focus on role-play.

P1 also noted that typical sessions were brief:

This informed our prototype design choice of administering the brief PHQ-4 form.

Two out of the four participants experienced the headset falling off their face, requiring constant adjustment and even manual holding to continue. P2 mentioned that

P4 felt similarly:

By P4’s session, the headset was also heating up, compounding discomfort.

All participants reported difficulty hearing the VP’s audio at maximum volume, requiring researcher clarification at multiple points.

P3 and P4 also reported difficulty understanding the VP’s accented speech at times, with P4 unable to comprehend certain responses even after replays.

P2 and P4 experienced technical glitches where the VP failed to respond to prompts, requiring researcher intervention to continue.

The overall impression of the simulation, particularly the emotional valence and level of satisfaction for the participants, was generally positive, consistent with the above-average overall PSSUQ usability score (mean 3.46, SD 1.71). The simulation’s realism allowed for participants to feel empathy toward the VP. Participants acknowledged the potential usefulness of the simulation in other contexts, and first-time AR and VR headset users also voiced some struggles.

Overall, participants found the prompts (Figure 7) useful in guiding them through the training. P3 appreciated that the prompts helped them focus on the specific goal at that certain time and that the prompts worked well in helping them follow the flow of the training. P4 mentioned how the prompts helped them to learn appropriate responses when they were unsure of how to respond. They told us:

They further mentioned that

However, navigating between 2 types of prompts (Figure 7) confused users and reduced immersion. Participants mentioned that having to pay more careful attention to the 2 different kinds of prompts broke the flow of immersion and took away from the experience. P2 elaborated that

P3 and P1 similarly felt that the simultaneous demands of attending to VP dialogue, body language, and PHQ-4 questions alongside varying prompt types created cognitive overload. P1 wanted more attention to the VP’s “body languages, whatever it may be, tone of voice, things of that nature,” hence the navigating between the 2 prompts did sometimes confuse them.

Participants pointed out that it was difficult to interpret qualitative responses and translate them into a quantitative scale (PHQ-4 form). P1 told us that

They further elaborated that they wanted to ask the VP more clarifying questions and have a more dynamic interaction to get the correct scale reading. P4 brought up that people usually don’t know how they feel, and it is MH workers’ jobs to be their advocate and interpret their response into a number and understand how to help them. Yet P4 expressed that

Participants found that using a real person’s voice with an accent increased realism, immersion, and hence the usefulness of training for specific populations or communities. P1 remarked that the VP’s accent is very realistic, which reflects people in the community having different accents that might be more difficult to understand. P4 echoed this and felt like the voice was real because it had an accent. P4 elaborated that

Although P2 felt that the voice was impactful, they mentioned, “I wasn’t sure how to respond, because I couldn’t understand. I think his accent, and also like the volume was low.”

All participants experienced difficulty with hearing due to low volume and accent. However, this motivated them to listen more intently during the training, and they expressed that it was beneficial to train for real-life situations in which some people are more soft-spoken. P1 raised that “it trains me to hear, so it does serve a benefit” echoing P2, who noted “in a real-life situation, when someone is speaking softly or in a different accent, I would need to be listening more intently.” P4 reiterated “I really had to process, really listen, which is good, I was really listening.”

Overall, participants highlighted that the body language of the VP was useful in training and increased realism, immersion, connection, and empathy. P1 mentioned that nonverbal cues were extremely helpful, for example, “the moving forward, the eyebrow expression.” P3 echoed the movement of the VP:

Similarly, P4 also expressed that they felt empathy and connection toward the VP, highlighting that

P4 further inferred from the movement that “He was afraid that I would tell others. So I think that’s why he leaned forward.”

On the other hand, there were mixed views on the realism of the VP. P4 mentioned that the distance of the VP in the headset facilitated easier conversation with the VP. They highlighted,

They also felt that the VP, being not too realistic and an animated avatar, made it easier to engage in conversation in a training setting. They added,

On the other hand, P2 mentioned

Hence, they requested more realism, specifically in the eye movements.

Although participants found that the pauses were good in giving space for validation and mimicked what a conversation would be like in the field, more interactivity in responses was desired from the VP. P2 felt that the pauses gave them a chance to acknowledge how difficult the conversation was and recognized that people would appreciate the space and pauses for their feelings to be validated. However, P2 highlighted that

P3 also echoed, “I was really interested in him elaborating more of what he was experiencing, and then reassuring him that it was okay.”

Traditional MH training often grapples with challenges in content simplification due to lack of resources [19,20]. Role-playing has often been used as a method to help trainees develop critical skills such as empathy, communication, and problem-solving [87]. While it is an effective way to simulate real-life scenarios and practice responses, it also presents challenges such as ensuring that role-plays are realistic and emotionally safe for participants, thus requiring skilled facilitators and significant preparation [87]. Echoing the literature, participants mentioned role-play in current training mostly being done between peers. Acting out more challenging scenarios, such as schizophrenia symptoms, was difficult to accurately role-play and might even be left out as peers were not trained actors. Current training often did not use trained actors as providers would require resources like time and money which they mostly do not have [88]. Our findings suggest that adding AR to the current workflow could increase more representative and effective role-play training.

Additionally, trainees may have theoretical knowledge but may require more intensive training in more nuanced practical skills such as reflecting on and interpreting people’s feelings and responses [89]. Participants emphasized the struggle of adapting theoretical knowledge to practice with current training methods. They expressed that theoretical knowledge is limited in its effectiveness, and they ultimately had to learn the ropes on the job and develop their own style of counseling. In addition, participants expressed difficulty when interpreting qualitative responses, translating them into a quantitative scale, and identifying how to help people due to individual subjectivity, reflecting their adaptability challenges with current traditional training. Notably, participants found that the AR training simulation allowed them to practice in a safe and comfortable environment where mistakes were acceptable and did not have detrimental consequences.

A systematic review found that racial and ethnic minority populations faced cultural MH stigma, which led to greater barriers to MH services and lack of trust [90]. Building trust between providers and patients in MH services is important to promote and maintain treatment and engagement [91]. Similarly, participants emphasized that building trust with patients was challenging but important. This type of engagement is difficult to train using traditional methods yet crucial for authentic patient relationships.

An integrative review found that realism and debrief were important in MH simulation training and improved trainees’ engagement and in bridging the gap between theory and practice [92]. Our study’s results highlight that the realism of the AR simulation and the quality of the animations led participants to feel empathy and connection toward the VP. Participants emphasized that they felt like they (the VP and participant) were the only ones in the room, indicating that the AR simulation was successful in immersing participants into a real-life scenario of conversing with a community member. The VP’s voice had an accent and was realistic, reflecting the population they would be interacting with, further allowing participants to feel like they were talking to a real person and practice communication. The VP’s animation and body language allowed trainees to further empathize with the VP and practice building trust. Through MH training simulation with SPs, trainees could gain empathy, compassion, and confidence, benefiting the people they serve [93]. The AR simulation was realistic, immersive, and sparked empathetic connections, thereby allowing trainees an environment to practice engaging and building trust in a realistic and effective manner.

Additionally, participants highlighted that AR simulation can better depict challenging scenarios, especially specific DSM categories like schizophrenia, depression, and anxiety. They emphasized that a standardized role-play training that best accurately depicts symptoms, rather than peers acting out the symptoms, would be extremely useful for practice. As such, having realistic practice in AR before real-life interactions can build provider efficacy and better care for patients. This kind of applied training is crucial for effective care by providers and MH task-sharers, with studies showing that using AR or VR to practice in a safe environment enhances knowledge and transferable skills, leading to the transfer of knowledge and skills in clinical practice [94]. Thus, this AR-assisted training tool could bridge the gap in training accuracy and enrich current traditional role-playing training.