The impact of COVID-19 on research within the homeless services sector

Abstract

Purpose

This paper aims to discuss the challenges of conducting research with homelessness services frontline workers during the COVID-19 pandemic.

Design/methodology/approach

Between 2015 and 2019, the research team surveyed frontline staff in three cities about their psychosocial stressors and needs. In 2020, the authors replicated the previous study and expanded data collection to seven cities across Canada to determine the extent to which the COVID-19 pandemic impacted the well-being of frontline staff. This report describes how the authors adapted the research methodologies to continue work throughout the pandemic, despite various restrictions.

Findings

The original studies had very high participation rates because of several methodological approaches that minimized barriers, especially in-person data collection. During the pandemic, distancing requirements precluded replication of these same methods. Research strategies that enabled staff participation during working hours, with designated time allotted for participation, was key for ensuring high participation rates, as access to technology, availability of free time and other factors frequently make online survey research a hardship for these staff. Restrictive interpretation and regional variations of COVID-19 guidelines by some research ethics boards were also a challenge to rapid and responsive data collection.

Originality/value

Few studies describe the experiences of frontline workers in the homelessness sector, and quantitative reports of their experiences are particularly scant. Consequently, little is known about specific methodologies that facilitate large-scale data collection in the homelessness services sector. The present research advances the field by providing lessons learned about best practice approaches in pre and post COVID-19 front line worker contexts. A strength of this research is the well-controlled design. The authors collected data within several of the organizations that had previously participated. This fortunate baseline provided opportunity for comparison before and during the pandemic; the authors can highlight factors that might have had influence during the pandemic.

Citation

Waegemakers Schiff, J., Weissman, E.P., Scharf, D., Schiff, R., Campbell, S., Knapp, J. and Jones, A. (2021), "The impact of COVID-19 on research within the homeless services sector", Housing, Care and Support, Vol. 24 No. 3/4, pp. 123-133. https://doi.org/10.1108/HCS-08-2021-0023

Conducting research on frontline homelessness sector workers has always been challenging. Staff are busy, stressed and not compensated well for their care and commitment. Workers may also prioritize client care above all other tasks. Research with this critical and difficult-to-access population has shed light on best practices in the field (Schiff and Lane, 2019). During the COVID-19 pandemic, however, researchers had to quickly pivot and adapt to changes in the worker population, their working conditions and the narrowing conditions under which we could conduct research. The purpose of this paper is to describe implementation challenges and solutions to studying large populations of homelessness sector frontline workers before and during the COVID-19 pandemic and beyond. To do this, we provide an illustrative example from a six-year, longitudinal study of homelessness sector frontline workers in seven cities across Canada.

The discourse on research infrequently focuses on implementation challenges. Furthermore, research on burnout, primary and secondary traumatic stress has often been impacted by recruitment issues and low responses rates (Schiff and Lane, 2019). Because of the COVID-19 pandemic, our team was presented with the unique opportunity to re-survey homelessness sector frontline staff working in pandemic conditions, who we had also surveyed prior to the pandemic, and measure the extent to which stressors created by the pandemic were affecting staff well-being. These additional stressors included ongoing threats of exposure to COVID-19 and increased demands from clients who feared contamination in shelters and client resistance to social isolation requests. While prior work had allowed us to identify challenges and opportunities specific to research in the field, the pandemic exacerbated these issues by making it even more difficult to reach and engage all staff. The lessons learned have broader implications for other field research.

Background

In both Canada and the USA, shelters are identified as high-risk work environments for multiple reasons: clients stay in crowded conditions with limited sanitation and live with multiple physical and behavioral (mental and addiction) issues that make them vulnerable to infectious disease (Buccieri and Schiff, 2016; Hwang and Burns, 2014). A scoping review of studies on staff burden in both Europe and North America found few studies that examine this population. This review suggested that staff in most countries are often untrained and ill-prepared for this work (Olivet et al., 2010; Schiff and Lane, 2019; Wirth et al., 2019). Emerging evidence suggests that frontline workers experience elevated risk of psychological injury from the stress of traumatic events experienced on the job (e.g. physical violence, suicide and overdose) (Schiff and Lane, 2019), and few organizations have implemented adequate trauma-informed practices (Waegemakers Schiff, 2019a). The significant increased risk of COVID-19 exposure adds to that stress. To develop policies that improve conditions for clients while protecting and enhancing worker safety, studies are needed that describe frontline homelessness sector workplace stresses, before and after COVID-19.

Homelessness sector frontline workers and workplaces: a study of pandemic preparedness in Canadian cities prompted by the alarm SARS viral outbreak in 2004, found that municipal governments, homeless organizations and staff in shelters were unprepared for the unique demands of those who could not “shelter in place” or practice preventative hygiene and physical distancing. Clients perceived staff as ill-informed on protocols for safety and how best to respond to client requests for accurate information about personal safety practices (Waegemakers Schiff and Lane, 2016b). Most of the learnings from that study failed to find traction among service providers who had neither the resources nor the initiative to develop comprehensive emergency plans beyond those for natural disasters, fires and floods. Resultantly, many organizations and municipalities did not have a comprehensive plan in place to inform homelessness sector staff of safety and protocols, in the event of a pandemic when the COVID-19 pandemic became a Canadian reality in March 2020.

From prior work, we know that when homeless services have not been prepared to address a pandemic, the resultant stress on staff can be enormous (Brown and O’Brien, 1998; Buccieri and Schiff, 2016; Waegemakers Schiff, 2019b). And there is limited information on how to support homeless sector frontline workers when workplace traumas occur (Olivet et al., 2010; Purtle, 2018; Wirth et al., 2019).

Prior to the COVID-19 outbreak, organizations providing homelessness services were ill-prepared to mitigate staff risk in high stress and traumatizing environments and the impact on staff was detrimental to their mental health and the services they delivered (Schiff and Lane, 2019; Waegemakers Schiff and Lane, 2016a). In the COVID-19 research project, we surveyed staff and added focused interviews with key administrators from organizations serving the homeless. Preliminary results confirm workers have faced additional work-related stressors and potential traumatizing situations in the workplace, including increased deaths from the virus, opioid overdoses and prolonged exposure to cold weather by many who prefer outdoor encampments to crowded indoor shelters.

Stress is socially contagious and readily transmitted among staff and clients (Dimitroff et al., 2017; Richter-Levin and Sandi, 2021) but is reduced by the presence of physical and psychological safety in the workplace. The COVID-19 pandemic has heightened concerns over lack of safety in homelessness sector programs as clients continue to be served in at-capacity shelters with large numbers of people forced to stay in close proximity, rendering physical distancing an impossibility. As well, proven hygiene options such as hand-washing and wearing protective masks and other personal protective equipment were initially hard to access. A correlate of increased workplace stressors is the high rates of sick leave and stress-related disability as reported by many organizations during key interviews with administrators. Furthermore, there are well-known workforce shortages of qualified, experienced frontline staff (Schiff and Lane, 2019). As a result, those who fall ill or require stress leave cannot readily be replaced, leading to a worsening care crisis for the remaining staff and their clients. Psychological trauma is highly impactful to victims and costly to organizations in terms of loss of productivity and prolonged stress leave (Milligan-Saville et al., 2017). The stressors specifically implicated by homeless sector staff have recently come to attention, as reports indicate rates of traumatic stress indicative of PTSD that are twice as high as other emergency services workers (Waegemakers Schiff, 2019b). The pandemic provided an opportunity to assess the extent to which staff stress had increased and which mitigating strategies might reduce this burden. However, recruitment and data collection strategies had to be modified to meet constantly evolving and locally divergent COVID-19 restrictions. Overall, we were concerned that participation rate, which had been strength of our previous research, would be diminished and the quality of survey responses would also be negatively affected.

Evolving methods for a dynamic research environment

Currently, we are conducting a study to examine work-related psychological injuries among homeless services frontline staff (HSFW) during the COVID-19 pandemic. The primary goal of the project is to understand the extent to which the COVID-19 pandemic impacted their psychological well-being and work-related disability. Study data derives from interviews and surveys from participating organizations across Canada and administrative data from the Alberta Workers Compensation Board (A-WCB). The findings aim to help organizations prioritize HSFW access to mental health services targeted trainings and other supports that would directly address their needs. An unanticipated consequence of the work has been an emerging understanding of the ways in which study protocols would need to be modified to maintain high rates of HSFW participation in the face of a changing and increasingly psychologically and physically dangerous working environment.

Research design: pre- and post-COVID-19: a major limitation of previous research on staff stress and burnout has been its reliance on convenience samples rather than population-wide groups. In a meta-analysis of STS, VC and burnout, Cieslak et al. (2014) noted that all reports consisted of convenience samples and most used online or mail in surveys, with return rates, where reported, ranging from 32% to 46%. Based on a report of psychosocial needs of frontline workers (Waegemakers Schiff and Lane, 2016a, 2016b), Statistics Canada produced a profile of this work force based on the 2016 Canadian census (Statistics Canada, 2019) which closely matched our results and provided strong external validity of data collected. This, in turn, supports the strength of our methodology.

As stress and PTSD lead to avoidance of related triggers, there is undoubtedly a selection bias in respondents who are coping with high stress levels are more likely to avoid additional stress or extra work required to complete a survey. The probability that those most impacted are least likely to participate in these surveys is quite high and results would skew results and underestimate true prevalence and severity. Our methodology aimed to avoid this limitation.

Over 85% of eligible organizations agreed to participate, and resultant data collection of surveys during work hours resulted in response rates varying from 85% to 100% in these organizations. Resultantly, we surveyed most eligible staff. In 2020, we applied for and received a grant from the Canadian Institutes for Health Research (CIHR) to assess COVID-19 specific effects on HSFW staff. The current study includes four additional cities across Canada and a qualitative component consisting of interviews with program administrators to assess organization-level responses. The plan was to replicate the prior research protocols with previously reported high participation rates for a robust result.

Survey recruitment and data collection: most quantitative studies which examine secondary traumatic stress and burnout in human services rely on convenience samples recruited from professional organizations or at training events (Molnar et al., 2020). This self-selection and the ensuing low response rates probably exclude staff most impacted by work-related stress, such as those too busy, tired or burned out to participate in data collection. In turn, this skewed participation could markedly understate the extent of traumatic stress in staff (Halbesleben and Whitman, 2013).

Previous work found that staff participation depended upon organizational leaders providing researchers with time and access to staff during regular working hours. These implementation details were deemed important and are detailed as follows: presenting the study rationale and procedures to staff at staff meetings; designated time for participants go to a private location to (if they chose) complete an anonymous survey; and return the survey in an unmarked envelop directly to researchers. This procedure ensures that the “testing environment”, within the workday, is similar across all sites while ensuring that workers who did not want to participate could not be identified. At the conclusion, a wrap-up and debriefing session for interested staff to discuss the study and the survey questions was offered. With organizational “buy-in” to the study, it was possible to facilitate a quick and easy data collection resulting in high participation rates, with, depending on site, between 85% and 100% of staff completing the survey.

Previous studies found that rates of traumatic symptoms are indicative of a PTSD diagnosis ranged from 33% to 41% (Schiff and Lane, 2019). Additionally, findings indicated that traumatic stress (i.e. still caring but struggling), and not burnout (i.e. struggling and no longer caring), was the predominant determinant of staff psychosocial distress. This was an important finding, as many interventions to mitigate burnout are not the same as those that can address needs of staff with trauma symptoms. By comparing with previous results, the present study allows us to examine increased rates of traumatic stress due to the pandemic. To ensure that pre- and post-pandemic data were comparable, we aimed to maintain fidelity to the protocols and response rates obtained in the original design.

Protocol modifications: protocol modifications were driven by the pandemic lockdowns and distancing restrictions which limited the mobility of our research team and access to staff within homeless service organizations. While an obvious solution might have been offering the HSFW survey through a secure online platform, we were reluctant and had previously avoided online surveys because of their notoriously low response rates (Shih and Fan, 2008) and consequent limited validity. As a compromise between safety and practicality, we adopted a two-pronged approach including both paper-based and on-line surveys, depending on the specific type of work assignment of staff in each program. Serendipitously, this allowed a comparison of differential response rates across two methodological approaches.

Specific data collection strategies for each organization were determined through discussions with executive directors or CEOs, which allowed them to decide on the most appropriate course of action. Most participating organizations have a combination of staff deployed to 24/7 shelters, street outreach, and support for those in transitional and “Housing First” programs. Shelter staff were required to continue to work on-site with clients. Those in outreach and support programs had latitude to work from home and plan in-person physically distanced meetings with clients. When possible, staff met with clients virtually and all staff meetings occurred over video conferencing. Staff with whom we could connect over a staff video conference meeting were scheduled for on-line surveys while staff who were primarily in shelters received paper-based surveys. Conflicting requirements for social distancing and working remotely impacted both HSFWs and research assistants (RAs) who were charged with coordinating the different streams of data collection while maintaining social distancing.

Paper-based survey administration: administration of the paper-based surveys was modified in several ways. Previously, research staff distributed the paper surveys. Unable to do so due to COVID-19 restrictions and to reduce perception of coercion on participants by supervisory staff, managers were requested to delegate distribution and collection of the paper-based surveys (in unmarked, sealed envelopes) to staff and to hand-off the surveys to a courier service. To ensure that surveys were distributed according to protocol, a project RA met virtually with HSFW to explain the study, reinforce the importance of procedures, and answer questions. For staff who could not meet with an RA, a senior project RA with COVID-19 quarantine shelter experience developed a short video to help the delegated HSFW complete the study tasks. The video addressed topics including the study rationale, objectives, and key ethical issues (e.g. confidentiality, voluntary participations, etc.). The video was distributed to all potential staff and managerial participants. To answer any questions HSFW may have had before initiating the online survey, the pre-recorded video was accompanied by research staff contact information.

As the pandemic unfolded and worsened, travel prohibitions precluded traveling to deliver or collect completed paper surveys. In some instances, major courier services refused to deliver or collect packages from high-risk places such as soup kitchens and homeless shelters. In some instances, alternative courier services organized by people with lived experiences of homelessness were able to transport the surveys. In others, shelter staff delivered packages of surveys to a central office acceptable to courier services for pick-up.

Internet-based survey administration: administrators at partner organizations reported that not all HSFW had access to computers at work and the means to afford personal computers or the internet at home. This digital divide precluded the opportunity for many shelter staff to complete on-line based surveys during working hours. In some instances, managers and supervisors created workarounds, such as providing staff the time to complete surveys on their mobile phones during work hours, or at the end of their shift, thus maintaining fidelity to survey protocols by completing the survey while at work.

For staff who worked remotely, researchers joined on-line staff meetings, explained the study, answered questions and provided a link to the on-line survey. By pre-arrangement, staff were then allowed to leave the video conference to complete the survey, and then return to the conference for a debrief and wrap-up. During the administration time, employers had no way of knowing if staff were completing the survey or declining to do so, thus ensuring the confidentiality of the participants.

Interviews: a second component of the study was to interview key administrators in each organization. Interviews are a standard feature of qualitative research and can be invaluable in understanding parallel quantitative data. Critical to the collection of in-depth, reliable data, is the context within which interviews are conducted, and researchers provide safe, comfortable, private and confidential location that follow interviewee preferences (Dempsey et al., 2016). These contextual parameters allow for the establishment of rapport and are valuable when the discussion veers to sensitive topics. When interviewing people in positions of authority, it is also paramount that the interviewer is able to probe beyond the public persona to explore the organizational dynamics that drive its operations. In-person interviews can achieve this through a delicate balance of interaction and empathy. However, when all interviews need to be conducted by video conference, these key dynamics are impeded by several factors, including the challenges of establishing a relationship that is mediated by video technology, limited exposure to the non-verbal communication, including body posture and positioning, lack of eye contact used to establish personal connections and difficulty in determining affective content (Weissman, 2013).

The literature on internet-based qualitative interviewing pre-dates the implementation of video conference technology (Kazmer and Xie, 2008) and does not address the extent to which the visual connect overcomes some of these challenges. Likewise, the literature on the effectiveness of telepsychology and video interviewing is emerging along with its use during the pandemic. Research on psychological services provided through video/distance technology suggests that it can be effective, allowing for reasonable visual and verbal communication of empathy, warmth, and understanding through matched speech, posture, and tone (Dempsey et al., 2016).

In the present study, we considered both the advantages and limitations of conducting all interviews over video conference technology. Video supported online research has been used in fieldwork among vulnerable and sensitive populations for many years (Pink et al., 2015; Weissman, 2017). Due to COVID-19, such virtual fieldwork is becoming more common. The immediate advantage is accessibility. In many instances, video conference technology also translates to a more casual ambience as people worked from home offices and were less constrained by office formalities. The realities that both interviewers and interviewees were facing common pandemic-related operational challenges created a common ground not easily found in situations of hierarchical imbalances of power when interviews occur in a formal office space. A final advantage to these video-based interviews was that some platforms provide a transcript of the meeting at its conclusion. This significantly reduced the time and cost burden of transcription, allowed ready access to the interview content and allowed interviewees opportunity to examine their conversation before formal inclusion in the analyses. In turn, this significantly shortened the time and effort to move to data analysis.

At the same time, conducting video-based interviews did create new challenges. Establishing rapport through a video call may only be partially successful, especially if this is a singular meeting rather than one of a series of conversations. There is a certain type of learning that takes place in dialogues and we call this dialogics. Dialogism is the process by which meaning is derived through conversation in various contexts, where power and status or other factors might influence the meaning of conversations. Recognizing that senior administrators carry certain authority and are often experienced in presenting to the public eye, senior members of the research team who themselves had management and leadership experience conducted these interviews. While this did not completely overcome the limitations of a video call, it allowed interviewers to recognize politically correct answers and those that did not fit with what was known about the organizational dynamics. The senior researchers had extensive interviewing experience in other contexts and were conversant in detecting subtle affective components which may be missed by less experienced persons. Thus, in the review of the transcripts, they were able to identify instances where the transcript did not convey the entire content or context of the issue at hand.

Research staff training: due to COVID-19-related restrictions, training of RAs also occurred online. We used video conferencing to bring cohorts of RA together to train them on all of the study protocols. A senior RA with working experience in shelters had participated in the previous study and provided illustrative examples to the trainee group. This RA also had a firm grasp of the data collection process at staff meetings and was easily able to adapt the revised protocols. Serendipitously, this RA was a trained social worker and also employed as a support worker in a COVID-19 quarantine hotel designated for individuals experiencing homelessness who had tested positive for the virus. This hands-on experience allowed the team to better appreciate the demands placed on frontline workers participating in the study. For example, our experienced RA described to the team how she observed that very few HSFWs were adequately trained to respond in a volatile work environment exacerbated by reactions of clients who were involuntarily detained because of their infectious status.

As this was a nation-wide study, recruitment and coordination of data collection strategies were greatly enhanced by the opportunity to produce and distribute a set of webinars that explained the background, importance of the study and methodologies. These were used across various regions and time zones with the options of offering live broadcasts to attract those who were interested in follow-up discussions. These webinars also paved the way for rapid dissemination of results, which can be done in waves rather than awaiting a final overall report, and was highly encouraged by the funders.

Data entry: a central database allowed for continuous monitoring of the survey responses. Paper surveys were scanned, and electronic versions were sent to RAs for data entry in an identical but separate database from the direct entry version used by staff working remotely. Thus, we could engage in systematic data management of errors/omissions which we expected to be somewhat different between the online and paper versions of the survey. This enabled the team to document the efficacy of online recruitment and participation, both with comparison to a paper-based cohort and also with comparison to results from the same study prior to the pandemic shutdown – an analysis that is new to the research literature.

Analysis of preliminary results includes 548 staff in over 40 programs in seven cities. In-person presentation (n = 210) and on-line (N-338) survey distribution resulted in response rates of over 80% of eligible staff while the online survey participation ranged from 50% to over 80%. Feedback provided indicated that in-person recruitment was favored by participants, some of whom had been included in a previous version of this study and commented on the value of researcher presence. A comparison of participant profiles in one city for which we had data from a similar study conducted in 2019 indicated that the demographic profiles of participants are the same.

Research ethics boards in COVID-19 times

During the pandemic, Research Ethics Boards (REBs) have also quickly refocused their attention toward COVID-19 and research participant safety, adding a new layer of consideration for universities and research staff. Indeed, the present study was based on existing REB-approved protocols, and in non-pandemic times, the new study would have been subject to a request for modification of an existing protocol, with changes including additional study sites and one additional widely used assessment tool. In Canada, research ethics is guided by the Tri-Council Panel on Research Ethics, which issued additional guidelines for research in the COVID-19 environment (https://ethics.gc.ca/eng/nr-cp_2020-09-02.html). The study covered three regions in Canada, and the team had to satisfy the REB of the principal investigator and the REB in each region. The differences in the interpretation of Tri-Council guidelines were apparent across the REBs. Despite substantial research evidence from past iterations of the study and other research that the survey and its imbedded instruments did not pose more than minimal risk, some boards refused to accept this view. Additionally, some REBs adopted a more stringent application of Tri – Council guidelines for determining whether a study is of more than minimal risk:

The probability of occurrence of the harm: This refers to the likelihood of participants actually suffering the relevant harms. An assessment of such probability may be based on the researcher’s past experience conducting such studies, on the review of existing publications that provide rates of the relevant harms in similar issues, or on other empirical evidence. And while researchers should attempt to estimate the occurrence of the relevant harms, this may be more difficult, or not possible, for new or emerging areas of research where no prior experience, comparable research or publications exist. (https://ethics.gc.ca/eng/tcps2-eptc2_2018_chapter2-chapitre2.html).

Ultimately, our study was deemed to include more than “minimal risk” to participants by a subset of REBs. The irony of this decision is highlighted in the fact that staff works in high-risk settings where trauma exposure is the norm for HSFW and they encounter greater risks in their jobs than the survey presented. This highlights the limitation of REB reviewers who have little first-hand knowledge of specific research environments.

Several aspects of these guidelines received additional attention: privacy and confidentiality issues involved in internet-based research, the need for RAs to connect with staff remotely rather than in face-to-face meetings and potential increase in vulnerabilities for some participants. Under this increased scrutiny, some REBs became much more vigilant of protecting research subjects, even when they do not belong to an identified vulnerable population. This included considerable discussion over the inclusion of a well-known and widely used instrument on adverse childhood experiences and supplemented existing (approved) questions about major adverse life events. The team perceived this as an intrusion into academic prerogatives over research methodologies.

We raise this issue to stimulate discussion about potential over-interpretation of the risk and harms section of the Tri-Council guidelines. One the one hand, we recognize that the pandemic creates real and severe risks for researchers and research participants. At the same time, research must continue during the pandemic – with reasonable protections in place – to ensure that organizations can create evidence-based, effective policies that mitigate pandemic risk now and, in the future. The net results of our discussions with multiple REBs was a prolonged delay in data collection and therefore access to results by agencies that wanted and needed the data to understand the ongoing support needs of staff.

Limitations

Although we created some economies of time and effort by using video conferencing technology in our study, centralizing online responses and the use of webinars and videos to explain and promote the study, our current response rate varies markedly between online and paper-based data collection and demonstrates that a lack of in-person contact has made recruitment a greater challenge. While a variety approaches have helped recruitment, this does not appear to replace the more personable activities that result when researchers are able to have in-person contact with key administrators and staff in establishing the research partnerships. Recruitment has been impacted by the lack of availability of agency participation during the most acute phases of COVID-19 outbreak when staff are overwhelmed with dealing with active outbreaks of the virus.

Another limitation of our current design is that we have fewer opportunities for student experiential learning than when our data was collected face-to-face. Students, who form the bulk of the data collection team, have worked remotely, which allows them a measure of freedoms and latitude, that can enhance skill-building, but reduces the mentoring that normally accompanies team discussions in a real-time environment. While we have built in debriefing time during the training and shadowing phase of the data collection, it does not fully compensate for the spontaneous discussion about research nuances that can be enriching for students.

Finally, this report focuses on a study that has not completed all of its data collection. There may be additional challenges that could still unfold. One of these emerging issues is the extent to which data collection has to stop during times of infection outbreak because staff are too overwhelmed with increased care duties that there is no appropriate time for them to complete a survey. A second consideration we are monitoring is the extent to which response rates are decreased because the ability for researchers to do in-person presentations of the study is constrained. These interactive discussions are critical not just for framing the study, but also emphasizing the importance and benefits of this research for the direct participants. Although the nature of the project requires that the research be conducted in the midst of this pandemic, the ever-changing circumstances have introduced real and evolving challenges.

Conclusion

Research in organizations serving individuals experiencing homelessness has often neglected the needs of staff who work on the frontlines. In many industries, the COVID-19 pandemic forced all levels of government and senior agency administrators to prioritize the welfare of staff and their needs leading to shutdowns. For some social services, this was simply not possible. The essential nature of the work completed by frontline workers in homeless organizations has led to awareness of the risks that staff encounter on a daily basis that have been exacerbated in the context of COVID-19. The added stress on an already over-burdened work force has been anecdotally reported. The research study that we described has provided an opportunity to learn more about how to conduct research effectively with this difficulty-to-study population, under typical circumstances and in pandemic-specific times.

As a team, assembling over vast distancing, without opportunities to establish working relationships in advance has been a challenge for some members. While the principal investigators had working relationships with other all other team members, they did not have this same connection with each other. The study notes that despite the utility of using video in some contexts, the nature of sporadic video conferencing can limit the opportunities for informal discussions and blurs the visual cuing and social lubricants that generate and stimulate innovations. Hence, the ability to incorporate exigencies into planning and problem solving can be seriously diminished. These adaptations likely translate to a decreased sense of familiarity and cohesion within the team.

From prior iterations of this study, the research team was aware of how to expeditiously solicit and get organizational support for study participation. Social distancing made recruitment more onerous as leadership in many organizations had changed in the last three years and this necessitated working out new connections in the identified organizations. Simultaneously, during outbreaks in respective communities, senior executives struggled with workloads that leave little room for inclusion of research, and simultaneously approached by multiple researchers eager to examine the impact of COVID-19 in various ways, which in turn makes recruitment a greater struggle.

We experienced efficiencies in data collection using on-line surveys that were electronically tabulated and video conferenced interviews that were instantly transcribed. The efficiencies were offset by the need to develop two parallel electronic based surveys, one for staff completion and the other for RAs to data enter paper-based surveys. The perceived efficiency of using an existing REB certification was off-set by the numerous and at times inconsequential requests from review boards that also did not have any experience in how to modify protocols in the face of pandemic distancing requirements. It is premature to conclude if these efficiencies outweighed the burdens of additional accommodations.

A frequently used expression is that the COVID-19 pandemic has provided opportunities to implement new ways of doing things in broad areas. We have explored how these allowed for opportunities and challenges in the context of doing research with staff. Because staff stressors and challenges depend on a full range of responses, the extent to which changes in data collection compromise the soundness of the data will depend on the extent to which this reduced the total participants. The lack of in-person contact in recruitment, RA training and team cohesion may be the most important limitations from adopting new protocols. A post-research analysis of the successes and the results will ultimately determine if research using on-line and social distancing approaches is sufficiently accurate to merit adoption as emerging protocols.