A content analysis of hospitality research's research methods in the 2010s

PurposeThis study aims to describe the development of hospitality research in terms of research methods and data sources used in the 2010s.Design/methodology/approachContent analyses of the research methods and data sources used in original hospitality research published in the 2010s in the Cornell Hospitality Quarterly (CQ), International Journal of Hospitality Management (IJHM), International Journal of Contemporary Hospitality Management (IJCHM), Journal of Hospitality and Tourism Research (JHTR) and International Hospitality Review (IHR) were conducted. It describes whether the time span, functional areas and geographic regions of data sources were related to the research methods and data sources.FindingsResults from 2,759 original hospitality empirical articles showed that marketing research used various research methods and data sources. Most finance articles used archival data, while most human resources articles used survey designs with organizational data. In addition, only a small amount of research used data from Oceania, Africa and Latin America.Research limitations/implicationsThis study sheds some light on the development of hospitality research in terms of research method and data source usage. However, it only focused on five English-based journals from 2010–2019. Therefore, future studies may seek to understand the impact of the COVID-19 pandemic on research methods and data source usage in hospitality research.Originality/valueThis is the first study to examine five hospitality journals' research methods and data sources used in the last decade. It sheds light on the development of hospitality research in the previous decade and identifies new hospitality research avenues.


Introduction
Hospitality research has developed rapidly in the 2010s. Changes in economic, social and technological factorsincluding postrecession economic recovery, growth of the sharing economy, globalization and evolution in technologyhave increased the number of hospitality research publications, changes in research topics and development in research methods and analytical techniques (e.g. Law et al., 2012). There is also an increased emphasis on research rigor with a few recent content analyses studies analyzing analytical techniques (e.g. Cheah et al., 2018;Xu and Martinez, 2018), research biases (e.g. Min et al., 2016;Y€ uksel, 2017) and bibliometrics (e.g. Cunill et al., 2019;Garc ıa-Lillo et al., 2016;K€ oseoglu et al., 2016;K€ oseoglu et al., 2015).
Despite these developments, much less attention in the last decade has been paid to examine hospitality empirical research methodsthe tools and means (e.g. experiments, Research method of hospitality research surveys and qualitative designs) researchers use to collect data to uncover new knowledge. A well-designed method ensures that hospitality research findings are not only valid but can also yield trustworthy practical implications to hospitality managers (McKercher, 2018). Accordingly, the research methods employed have often been suggested as an indicator to assess maturity, paradigm, sophistication, quality and progress of a discipline (e.g. Baloglu and Assante, 1999;K€ oseoglu et al., 2016;Rivera and Upchurch, 2008). However, the lack of recent content analysis of the research methods used in hospitality research hinders our understanding of the current state of hospitality research or ways to improve its sophistication level. Indeed, the recent development of technology has facilitated new research methods, such as user-generated data (e.g. online review), web data (e.g. hotel pricing data on websites) and simulation (Stamolampros et al., 2020;Lu and Stepchenkova, 2015). Compared with traditional research methods (e.g. survey and experiment), these new methods are available to hospitality managers and researchers at a low cost (Lu and Stepchenkova, 2015). Technology also facilitates new data sources (e.g. online panels) (Aguinis et al., 2021). More importantly, hospitality is a broad field with different functional areas (including marketing, finance, strategy, operations, technology and human resources [HR]) covering multiple geographic regions (including North America, Europe, Asia, Oceania, Latin America, Africa and multiregion) (Baloglu and Assante, 1999;Shen et al., 2018). Yet, the education and preference in research methods in each function area and geographic region can differ (Ali et al., 2021;Amarante, 2014;Mehmetoglu, 2004;Shen et al., 2018). Therefore, a description of research methods and data sources by each functional area and geographic region can enhance validity and pinpoint specific areas that need further development (Shen et al., 2018).
As we progress to the new decade of the 2020s, it is time to take stock of the research method used in the 2010s and look forward to new opportunities. To ensure the findings can be comparable to the content analysis of research methods conducted by Baloglu and Assante (1999), we focused the content analysis on the five established hospitality journals, including the Cornell Hospitality Quarterly (CQ), International Journal of Hospitality Management (IJHM), International Journal of Contemporary Hospitality Management (IJCHM), Journal of Hospitality and Tourism Research (JHTR) and International Hospitality Review (IHR, formerly the FIU Hospitality Review). This content analysis focused on the research methods and data sources used in original hospitality empirical articles published in these five journals in 2010-2019 by (1) time span, (2) functional areas and (3) geographic regions of the data. It provides implications on the development of hospitality research in the last decade, identifies new avenues of study designs for functional areas and sheds light on potential collaboration opportunities and education and training for academia in different geographic regions. In the next section, we present a brief review of content analyses within the boundaries of hospitality.
Historically, survey research has dominated hospitality research (Baloglu and Assante, 1999). From 1967 to 2005, there were increasing empirical research articles over time, with 60-80% of the empirical hospitality publications using surveys (Baloglu and Assante, 1999;Chon et al., 1989;Rivera and Upchurch, 2008). Such findings were in line with Dolnicar (2018), which showed the popularity of surveys in high-impact hospitality research in 2017. However, field development, editorial decisions and calls from prolific scholars can change the research method usage patterns. For example, former CQ Editor Michael Lynn has called for mix-method studies to "establish the generalizability of effects" (Lynn, 2017, p. 228). This echoes with the development outside hospitality (e.g. information systems, organizational development and strategic management), which requires multiple studies to validate the findings (Elbanna and Gherib, 2012;Olsen, 2019;Venkatesh et al., 2013). Similarly, there were rekindled interests in qualitative studies, as evidenced by calls from scholars (e.g. Arendt et al., 2012) and the creation of a special qualitative column in IJHM in 2018 (Elsevier, 2017). There were also calls for experiments as a tool to show causal relationships (Fong et al., 2016;Mattila, 2004). Recent technology development has facilitated new research methods (e.g. user-generated data, web data and simulation), reducing reliance on surveys (Lu and Stepchenkova, 2015). For example, user-generated data dramatically increased from 1 article in 2001 to 33 articles in 2012 (Lu and Stepchenkova, 2015). Accordingly, hospitality research method usage might have changed considerably since the last review in the 1990s (Baloglu and Assante, 1999, which revealed the research patterns in five hospitality journals as in this study) and 2000s (e.g. Rivera and Upchurch, 2008, which focused on IJHM articles). In short, we ask RQ1. What were the research methods used in the 2010s?

Data sources
Although each research method diverges in its ability to test causal relationships (Baloglu and Assante, 1999), data sources and samples also impact the external (i.e. generalizability and realism) and internal validity (i.e. measurement precision) of the findings. The strengths and weaknesses of each method are summarized in Table 1. However, such discussions are based on assumptions of the use of specific data sources. For example, Baloglu and Assante (1999) suggest that survey research can have high external validity if researchers use an effective sampling strategy. Similarly, because it is generally assumed that experiments use convenience samples with scenario experiments, scholars argue that experiments demonstrate causality at the cost of realism (Fong et al., 2016;Mattila, 2004). With the assumption of using a small sample based on personal contact and snowballing sampling techniques, qualitative studies are commonly touted for providing rich data with a high level of realism at the cost of measurement precision and generalizability (Arendt et al., 2012;Walsh, 2003). Archival studies mostly use publicly available data, which results in a high level of context realism and a low level of measurement flexibility (Jones, 2010).
However, technology facilitates the use of online research panels (e.g. Amazon's MTurk) and video conferences (e.g. Zoom), which lowered the cost for both quantitative (Stritch et al., 2017) and qualitative studies (Richard et al., 2018). It also changes previous assumptions of data sources used for specific research methods. For example, although online panels (e.g. MTurk) provide cheap and diverse data sources that can be used for both surveys and experiments, they may not represent the population (Aguinis et al., 2021). There are also  Table 2 lists the major data sources used in hospitality research. Given the changes in data sources, this study asks RQ2. What were the data sources used in the 2010s?
Research methods and data sources by functional areas To provide a refined understanding of research topics and methods used, some researchers focused their content analysis on one functional area within hospitality researchmarketing (e.g. Line and Runyan, 2012;Oh et al., 2004;Yoo et al., 2011). These studies found that most marketing studies published between 2000 and 2010 use empirical field surveys. The results were in line with Morosan et al. (2014), which suggested a paradigm shift in hospitality marketing research in 1989-2013 to focus on empirical research with a change in the research method. Unfortunately, content analyses including a method component are rarely seen in other functional areas, including finance, strategy, operations, technology and HR. Since each functional area has its research evolution, boundaries and field/disciplinespecific expectations (Henning et al., 2005), a content analysis of hospitality marketing research may not inform the general development of hospitality research. Aside from differences in norm and tradition developments, functional areas may use different research methods and data sources because of data availability and measurement levels (Li and Ryerson, 2019). For example, studies focusing on strategy and finance study organizational profitability as outcomes may need to rely on organizational level data, making it difficult to rely on research methods that use primary data (e.g. experiments and surveys) (Li and Ryerson, 2019). It suggests a need to extend Baloglu and Assante (1999) and Rivera and Upchurch (2008) by comparing the research method and data sources used in the different functional areas in the 2010s.

RQ3
. What were the research methods and data sources used in each functional area in the 2010s? However, as globalization increases in the 2010s, there is rapid development in the hospitality industry in Africa and Latin America (Benzitouni and Wawira, 2019;Schmitt and Cruz, 2018). As both Law et al. (2012) and Shen et al. (2018) focused on research published before 2011, it can be interesting to assess whether hospitality researchers have taken advantage of the globalization trends and collect data from different geographic regions.

RQ4.
What was the geographic focus of data collection by hospitality researchers in the 2010s?
Research methods and data sources per geographic regions Due to differential development, geographic representation provides better insights into assessing the evolution of research (cf. Law et al., 2012;Shen et al., 2018). Some studies focused on the content analysis in a specific country or geographic region, arguing that language barriers limit the exchange of information and research approaches among international scholars (Evren and Kozak, 2014;Huang and Hsu, 2008;K€ oseoglu et al., 2015;Mehmetoglu, 2004;Tsang and Hsu, 2011). Education and linguistics may also influence the changes in the method used (Shen et al., 2018). For example, non-English-native speakers may prefer sending quantitative over qualitative studies to the five studied journals because of the difficulties translating participants' in-depth responses (Van Nes et al., 2010;Rahman, 2020). Data availability also varies across geographic regions (Amarante, 2014). This is particularly true with the use of panel data. For example, MTurk has been overrepresented by US and Indian users (O'Brochta and Parikh, 2021). At the same time, the online panels are rarely represented in geographic regions with poor Internet access, such as Africa and Latin America (Amazon Mechanical Turk, 2017). Therefore, we take a global focus to extend these studies concentrating on a single region/country (Evren and Kozak, 2014;Huang and Hsu, 2008;Mehmetoglu, 2004;Tsang and Hsu, 2011) and compare the research method and data sources in different geographic regions.

RQ5.
What were the research methods and data sources used in each geographic region in the 2010s?

Journals and articles selection
This study analyzed the research method used by the articles published in the five hospitality journals (CQ, IJHM, IJCHM, JHTR, IHR) from 2010 to 2019. In this section, we describe how we gathered data for the study. We first define our research scope. Next, we describe the coding methods. Based on the above discussions, we ask five questions, including RQ1. What were the research methods used in the 2010s?
RQ2. What were the data sources used in the 2010s?
RQ3. What were the research methods and data sources used in each functional area in the 2010s?
RQ4. What was the geographic focus of data collection by hospitality researchers in the 2010s?
RQ5. What were the research methods and data sources used in each geographic region in the 2010s?

Research method of hospitality research
Consistent with Baloglu and Assante (1999), we narrowed our content reviews in these five hospitality journals (CQ, IJHM, IJCHM, JHTR and IHR) for four reasons. First, because tourism and hospitality research differ in the unit of analysis and the data availability, we limited our scope to hospitality research and excluded tourism journals (e.g. Annals of Tourism Research, Tourism Management). Second, we focused on hospitality journals with a general focus and excluded some functional-specific journals, such as the Journal of Hospitality and Tourism Education and the Journal of Hospitality and Tourism Technology.
All five journals specify that they welcome research from all functional areas, such as marketing, finance, HR, strategy, operations and technology. Third, all the journals are wellestablished. Using the same set of journals by Baloglu and Assante (1999) also allows a direct comparison of the results in this decade (2010-2019) and the 1990s (1990)(1991)(1992)(1993)(1994)(1995)(1996). Finally, all of these five journals welcome submissions of all research methods.

Coders and validity of coding
We analyzed the content systematically. First, three researchers developed the coding scheme in an initial meeting. Then, to ensure the accuracy and validity of coding, two researchers coded each article. Finally, inconsistencies were resolved by a discussion among the three researchers.
Coding method Empirical vs conceptual. Researchers manually coded the articles in five steps. In the first step, we identified the articles to be analyzed and only included original empirical hospitality articles and research notes. Excluding editorial, there were 3,227 articles published between January 2010 and December 2019 in the five hospitality journals. After excluding 48 research method papers, 110 destination tourism papers, 199 literature reviews and 108 conceptual papers, we retained a total of 2,759 original hospitality empirical articles for this content analysis.
Coding research method and data sources. In the second step, we coded the research method into (1) mixed method, (2) survey, (3a) scenario experiment, (3b) field experiment, (3c) quasi-experiment, (4a) structured questionnaire, (4b), interview, (4c) focus group, (5a) printed materials, (5b) public data, (5c) website data, (5d) user-generated data, (6) objective data and (7) simulation based on the definition in Table 3. We also coded the data sources into (1) student data, (2) panel data, (3) organizational data, (4) user-generated data, (5) public data, (6) personal contacts, (7) random data and (8) mixed data sources based on the definition in Table 2. Finally, we noted the number of empirical studies in each article. Most articles (2,628) have one study, and 131 articles have multiple empirical studies. Among the 131 multi-studies articles, 50 articles used mixed methods, and 81 used the same methods. Additionally, 20 of the 131 multi-studies articles used mixed data sources, while the other 111 used the same data sources.
Coding functional areas. In the third step, we identified the functional areas. Following Baloglu and Assante (1999), we classified articles into six functional areas: marketing, finance, strategy, operations, technology and HR. Table 4 lists the definitions of each functional area, the editors' functional backgrounds and the distribution of articles. To ensure consistency in coding and avoid overlaps in functional areas, we coded the functional areas based on the study's dependable variables. For example, a study investigating the factors that increase customer satisfaction (e.g. Choi and Chui, 2001) was considered a marketing study. In contrast, a study investigating factors driving employee behaviors (e.g. Curtis et al., 2013) was classified as a human resources study.
Coding geographic regions. Fourth, we coded the studies' geographic regions. We manually coded regions based on major continents: Africa, Europe, Asia (which includes IHR Russia, East Asia, South Asia and Middle-eastern), North America (includes Canada and the USA), Latin America (which includes Mexico, South America and the Caribbean), Oceania (which includes Australia and New Zealand) and multi-regions (which collect data from two or more regions). Considered similarities in culture within a large continent area (Hofstede, 2001) and some studies collecting data from multiple proximal countries, regions can provide more clarity in the results than a country-level analysis. In the next section, we present the results of our content analyses.

Research methods Definitions
Mixed method Research that uses two or more research methods Survey Research that uses scales to measure respondents' and raters' perceptions, attitudes and behaviors Experiment: scenario Research that randomly assigns functionals into experimental conditions and asks respondents to react based on a fictional case, plot and scenes Experiment: field Research that randomly assigns functionals into experimental conditions and asks respondents to react based on experience Experiment: quasiexperiment Research that assigns functionals into different conditions and asks respondents to react based on the conditions without random assignment Qualitative: Structured questionnaire Research that uses a list of pre-set open-ended questions to understand respondents' attitudes and behaviors; all participants receive the same list of questions and respond individually Qualitative: interview Research that asks respondents questions on a topic area in an unstructured one-on-one format; researchers can ask follow-up questions Qualitative: focus group Research that asks a group of respondents questions on a topic area and allows the group to discuss a topic Archival: printed materials Research in which researchers code data using news articles and library resources on a particular case Archival: public data Research that uses archival, publicly available data collected from financial institutions, annual reports and government data Archival: website data Research that uses website meta-data (e.g. time spent on the website, hit count, website traffic) and/or data available on a website (e.g. hotel price, locations) Archival: user-generated data Research that collects data users inputted content (e.g. online review)

Objective data
Research that observers keep track of the data in the respondents' natural environment using machines and company records (e.g. air quality index, heart rate, slot machine pay rate) Simulation Research that uses computer-generated data and data modeling techniques to test model Note(s): Please see Baloglu and Assante (1999), Arendt et al. (2012) for full discussion Research methods' usages in the 2010s RQ1 focused on the research method used in the 2010s. Table 5 illustrates the research method used in the last decade. Survey research increased sharply in the early 2010s from 44% in 2010 (78 articles) to 55% in 2013 (152 articles), followed by a stable period in the rest of 2010s ranging from 51 to 48% in 2014-2019 (128-237 articles). Overall, 51% of analyzed articles used survey designs (i.e. 1,401 articles).
Archival data were the second popular research method (593 articles, 22% of all articles), comparable to the results in the 1990s (19%, Baloglu and Assante, 1999). While the use of public data, including data from financial markets and census data, decreased to 13% (363 articles), new forms of archival datanamely web data (3%, 79 articles) and user-generated data (4%, 116 articles)played a more prominent role in empirical hospitality research. Although archival data remained relatively stable over time, there were some changes in the sub-categories. While the use of printed materials and public data decreased over time, there were more published articles using web data and user-generated data. In particular, the number of studies using user-generated data, such as online reviews, increased from 3 articles (2% of the published articles in 2010) to 42 articles (9% of the published articles in 2019).
Experiments were the third popular research method (11% of all articles, 318 articles). There was also a strong positive correlation between the total number of studies and the percentage of the experiment study (r 5 0.69, p < 0.05). Scenario experiment was the most popular among the three different types of experiments (8%, 225 articles), followed by quasi-experiment (2%, 56%) and field experiment (1%, 37 articles). Specifically, the representation of scenario experiments doubled over the years (from 4% [7 articles] in 2010 to 8% [41 articles] in 2019).
Around 9% of the published articles (235 articles) used a qualitative method. Among all three qualitative methods (structured questionnaire, interview and focus group), interviews were the most popular, with 200 published articles (7% of all articles). The number of qualitative studies decreased in the first half of the 2010s, dropping from 27 articles (16%) in 2010 to 14 articles (6%) in 2014. However, the number of qualitative studies increased in the latter half of the 2010s, representing 9% of the articles published in 2018 (29 articles) and 2019 (42 articles).
Similar to previous studies (e.g. Baloglu and Assante, 1999;Law et al., 2012), there was only a handful of studies using objective data (1%, 40 articles) and simulation (1%, 14 articles). Around 6% of the published articles used a mixed method design. However, the portions of mixed method articles decreased from 10% in 2010 (18 articles) to only 6% in 2019 (29 articles). The total number of publications published in a year was negatively related to the percentage of mix-method studies (r 5 À0.56, p < 0.1). Figure 1 answers RQ2, which investigates the use of data sources in the 2010s. Half (50%) of the studied articles use organizational data (1,368 articles), making it the most used data source. However, organizational data usage decreased from 58% (104 articles) to 35% (174 articles), while panel data increased from 3% (5 articles) to 24% (118 articles) over time. Although public data dropped from a high of 21% in 2010 (38 articles) to 12% in 2016 (32 articles), it slowly recovered to 18% in 2019 (93 articles). User-generated data increased from 1% (2 articles) to 8% (40 articles ). Other data sources, including personal contact (2%, 44 articles), random data (4%, 106 articles) and mixed data sources (5%, 125 articles), remain to be used in less than 5% of articles in 2010s.

Functional areas
To further understand the above results, we analyzed how the functional areas of original hospitality empirical research changed over time (see Figure 2). Marketing was the most popular functional area (40%, 1,102 Figure 3 addresses RQ3 by showing the research methods by functional areas. Functional areas were related to research method usage (χ 2 df 5 65 5 039.05, p < 0.01). Operations was the only functional area that used all seven research methods, including 29% surveys (70 articles), 27% archival data (65 articles), 19% qualitative methods (47 articles), 9% objective data (22 articles), 6% experiments (15 articles) and 3% simulations (7 articles). Additionally, 7% (18 out of 244) of the operations articles used mixed method designs.

Research methods by functional areas
Similarly, marketing articles used a wide range of methods, including 5% mixed method (60 articles), 53% surveys (584 articles), 24% experiments (260 articles), 12% archival data (133 articles) and 5% qualitative methods (51 articles). The increasing number of marketing studies over time (see Figure 2) and the diverse methods used in marketing articles (see Figure 3) may explain the rise of experiment and user-generated data over time. Indeed, 83% (260 out of 315) of the experimental studies and 80% (93 out of 115) of the user-generated data were marketing articles.
The diversity of research methods used in operations and marketing articles strongly contrasts with finance articles, which predominantly used archival data (84%, 261 articles). Moreover, 73% (227 articles) of finance articles use only one category of archival datapublic data. Indeed, 63% (227 out of 360) of the public data articles were finance articles. A small portion of the finance articles used surveys (8%, 26 articles) and qualitative methods (2%, 6 articles). Around 3% of finance articles (10 articles) used a mixed method design. Only one finance article used experiments, making it the field with the fewest articles that employed experiments.
Similarly, HR articles did not use diverse research methods. Over 76% of HR articles (551 articles) used surveys, while 11% (78 articles) used qualitative methods. With only 5% of HR articles (35 articles) used experiments, experimental designs were not common in HR articles. There was only 5% (35 articles) used a mixed method design. Only 4% of HR articles (29 articles) used archival data. There were no HR studies that used objective data, and only 1 HR study used simulations. Data sources by functional areas As shown in Figure 4, functional areas of research were related to the data sources. With marketing research had a good mix of different research methods, it also used diverse data sources. Organizational and panel data were most popular, accounting for 46% (503 articles) and 24% (266 articles) of marketing articles, respectively. However, marketing articles also use other data sources, including public data (7%, 82 articles), user-generated data (7%, 73 articles), student data (5%, 57 articles) and personal contact (1%, 13 articles). There were also 4% of marketing articles (46 articles) that used mixed data sources. Organizational data were most commonly used in HR studies (74%, 543 articles). Around 8% of HR studies (59 articles) used panel data, 5% (38 articles) used student data and 5% (36 articles) used public data. However, other data sources, including random data (1%, 9 articles), personal contacts (1%, 8 articles) and user-generated data (1%, 4 articles), were seldomly used. Only 4% of HR studies (32 articles) used mixed data sources.
Geographic regions overtime RQ4 investigates the research intensity in different geographic regions. Despite increased emphasis on globalization, Table 6 showed that the portion of studies conducted in the various geographic regions remained relatively stable over time. Results showed that most articles used data from North America (1,119 articles, 41%) and Asia (979 articles, 35%). European data articles increased slowly from 12% (22 articles) in 2010 to 16% (77 articles) in 2019. Overall, there was 14% of the articles used European data (387 articles). Unfortunately, there was only a handful of studies using data from Oceania (3%, 94 articles), Africa (1%, 37 articles) and Latin America (1%, 19 articles) in the 2010s. Less than ten articles were from Latin America and Africa each year, representing around 0-2% of the analyzed articles. There was a drop in multi-region articles from 2010 (6%, 10 articles) to 2014 (2%, 6 articles). However, the number of multi-regions articles increased steadily since 2015 (4%, 11 articles). As a result, 7% of the 2019 articles (33 articles) used data collected from two or more geographic regions. Figure 5 answers RQ5 by showing the research method used in different geographic regions. Geographic regions had different research method usage patterns (χ 2 df 5 91 5 572.74, p < 0.01), suggesting that the usage of hospitality research can differ across geographic regions. It is worth noting that user-generated data were collected from all six geographic areas and contributed 10% of the multi-region articles (13 articles).

Research methods by geographic regions
Researchers used a variety of methods to collect data from North America. It had a low reliance on survey research (40%, 453 articles) and has a good mix of experiments (20%, 223 articles) and archival data (28%, 311 articles). It also used all other methods, including qualitative studies (5%, 56 articles), objective data (2%, 25 articles) and simulation (1%, 5 articles). Indeed, researchers collected 62% (223 out of 360) of the public data studies from North America.
Despite the relatively low usage of experiments (7%, 26 articles), data collected from Europe originates from a good mix of methods, including survey (43%, 168 articles), archival data (29%, 114 articles) and qualitative studies (14%, 54 articles). Furthermore, it also contributes to the usage of mixed method (5%, 19 articles), objective data (1%, 4 articles) and simulation (1%, 2 articles). Thus, the results indicating that the development of the research methods was quite sophisticated in Europe. Interestingly, Oceania had the lowest reliance on surveys (37%, 35 articles) and the highest reliance on qualitative studies (25%, 24 articles). Similar to North America and Europe, Oceania also had a good balance of experiments (13%, 12 articles), archival data (16%, 15 articles) and objective data (3%, 3 articles).
Although Asia also used all research methods, it relied on surveys (67%, 660 articles). On the other hand, there was a relatively small portion of archival data (12%, 114 articles),   , objective data (1%, 4 articles) and simulation (1%, 2 articles). Such reliance on survey research was similar to other developing regions, such as Latin America (53%, 10 articles) and Africa (73%, 27 articles). Researchers collected archival data (21%, 4 articles) and a mixed method study (16%, 3 articles) in Latin America. On the other hand, some qualitative studies were collected in Africa (16%, 6 articles). Only one qualitative study (5%) and one simulation (5%) were collected from Latin America. Similarly, researchers only collected one mixed method study (3%), one objective data (3%) and two archival data (3%) in Africa. However, no experiment was collected from Latin America or Africa. Table 7 illustrates the data sources by geographic regions. Although North America and Asia contribute the most data, the types of data they contribute can be very different. For example, while North America contributes the most student data (64%, 69 out of 107 articles), panel data (76%, 300 out of 397 articles) and public data (52%, 257 out 499 articles), Asia contributes the most organizational data (50%, 687 out of 1,368 articles), personal contacts (59%, 26 out of 44 articles) and random data (48%, 51 out of 106 articles). Of the 133 studies using user-generated data, 46% originated from North America (52 articles), 19% from Europe (22 articles), 18% from Asia (20 articles), 4% from Oceania (4 articles), 2% from Latin America (2 articles) and 1% from Africa (1 article). Researchers also collected 11% of the usergenerated data from multi-region (11 articles), indicating that it can be possible to conduct multi-region research. In the next section, we discuss the implications of the result of this study and make recommendations for the journal editors, researchers and institutions. Additionally, we address the limitations of this project.  As technology had enabled new research methods (e.g. user-generated data) and new data sources (e.g. online panel data), there was growing diversity in the research methods employed. There was a slight increase in qualitative research in the mid-2010s. This change could be attributed to the launching of designated special qualitative sections in IJHM in 2018 (Elsevier, 2017), indicating that journals' special issues can impact researchers' method choices. Recent calls of specific research methods, such as experiments (Fong et al., 2016) and user-generated data (Lu and Stepchenkova, 2015), increased the usage of these methods and decreased reliance on the survey. However, as in the 1990s (Baloglu and Assante, 1999), objective data and simulation remain minimal, suggesting potential new avenues. Thus, the research method and data sources usage depend on journal editorial decisions and the calls from prolific scholars.

Data sources by geographic regions
Overall, these results suggested that hospitality research in the 2010s was more advanced and diversified than in previous decades. The portion of survey studies across the 2010s was significantly lower than previously reported (e.g. Baloglu and Assante, 1999;Law et al., 2012). Compared with Baloglu and Assante (1999), which use the same list of five journals, the reliance on surveys dropped from 80 to 51% during the period of 1990-2010s. However, surveys remained to be the most popular research method in the 2010s. While hospitality research was using the new technology-enabled method, such as website data and usergenerated information, the use of objective data and simulation remains minimal.
However, the technology also results in new threats, as seen in the decreased usage of organizational data and increased reliance on panel data. One possible reason is the availability of online panels, such as Amazon MTurk, Qualtrics Panels and SurveyMonkey online panels. Although panel data provide easy access to the diverse sample (MTurk, 2017), organizational data can provide richer contexts for researchers to understand the implications of the studies.
The developing trend of using diverse methods was not the same in all functional areas. Similar trends can be seen in the data sources because data sources and research methods are highly related. Together, the results showed that functional area was related to the research method used, and each functional area varied in terms of developments in research techniques. Unfortunately, the results indicate that some functional areas develop faster than other functional areas. While marketing and operational research contribute to the diversity of the research method used, other functional areas, such as HR and finance, continued to use the same method (survey with organizational data; archival data with public data) as in the 1990s. Because finance and HR were the second and third most popular functional areas, the primary usage of archival data in finance articles and surveys in HR articles may explain the popularity of survey and public data in all hospitality research.
Compared with Shen et al. (2018), our results showed that the geographical landscape of hospitality research did not change much over the yearsthe majority of the studies used data from North America and Asia. Oceania, Latin America and Africa remained underrepresented in the five hospitality journals focused on this research. The results IHR indicate that the findings yield in hospitality research from these five journals may be more applicable to North America and Asia. However, the findings from these five journals may have limited generalizability to developing geographic regions, such as Oceania, Africa and Latin America. One plausible explanation would be that most established hospitality schools are in the USA, Asia, and Europe, and all five journals are US-based. Geographic regions were related to the research method used, which may be attributed to the training and the differential norms of the preferred method in the culture (Shen et al., 2018). Geographic regions also provide different data sources. Aside from language and training, the differences across geographic regions can be attributed to legal requirements (Benoit et al., 2019). For example, while getting data from organizations in North America requires corporate legal approval, getting such data in Asia is usually a regional office decision (Benoit et al., 2019). Similarly, although most North American publicly traded organizations are required to share their financial information, the sharing requirements can be different in Asia (SEC, 2020). However, user-generated data across all geographic regions suggest that this new research method can be a potential avenue for collecting data in different geographic regions, which may be difficult for traditional research methods.
Finally, due to the difficulties of using a mix-method or collecting data from multiple geographic areas, only a small portion of articles used mixed method (6%, 158 articles) or collected data from multiple geographic regions (4%, 124 articles). Unfortunately, this suggests that the hospitality research method has not improved significantly since previous content analyses work in the past two decades (e.g. Baloglu and Assante, 1999;Law et al., 2012). Therefore, as researchers, we need to spark new conversations by diversifying our research methods and study designs. Drawing on the above findings, we made a few recommendations to the hospitality editors and researchers.

Recommendation for editors: Be open to nontraditional research methods
To stimulate research method diversity, editors should be more open to other research methods that have seldom been used in the functional areas. For example, several mainstream studies suggested that experimental designs can be utilized to extend finance (e.g. Libby et al., 2002;Karlan, 2005) and HR research (e.g. Aguinis and Bradley, 2014;Podsakoff and Podsakoff, 2019). Therefore, we encourage editors to create a special section on mixed method articles. They can also call for nontraditional methods in special issues (e.g. nonsurvey research in HR or nonarchival data research in finance), thus allowing for exemplar studies that show interesting and potentially different results by using unconventional methods.

Recommendation for researchers: use new tools to conduct research
It is important to note that each research method diverges in its generalizability, realism and measurement precision (Baloglu and Assante, 1999). Thus, we reiterate Baloglu and Assante's (1999) recommendation for researchers to use multiple research methods in their studies. Specifically, we encourage researchers to consider adding qualitative analysis into quantitative research. For example, researchers can use a qualitative study to showcase the decision-making process and explains quantitative results. We also encourage researchers to use more qualitative research in finance and marketing.
Technology also enables researchers to have more research methods to conduct research. Our results showed that the utilization of user-generated data was on the rise. However, this rise was mainly limited to marketing research. Researchers from other functional areas can learn from their marketing peers and use more user-generated data. For example, HR researchers can use online employee reviews (e.g. Glassdoor and Indeed) to understand employee outcomes (Stamolampros et al., 2020). Similarly, researchers can use traditional Research method of hospitality