Development of a Thai gaming disorder scale for children and adolescents

Abstract

Purpose

This study aimed to develop the Thai gaming disorder scale (T-GDS) in children and adolescents to serve medical staff and investigate the effectiveness of the scale.

Design/methodology/approach

This is a research and development study. In total, 217 participants were children and adolescents between 8–18 years, then classified into four groups according to Children's Global Assessment Scale (CGAS). The T-GDS was developed; its content validity was then investigated by three experts. Mock assessment was conducted on 15 individuals replicating the actual sample group before the assessment was tested on the sample group by two medical staff. The quality of the scale is assessed through reliability, validity and cut-off point analysis.

Findings

Exploratory factor analysis (EFA) extracted four components with 18 items meeting the criteria and have Cronbach's alpha of 0.95. The analysis of ROC curve, to determine the cut-off point, associated the mild game addiction group with T-GDS score = 14; moderate group score = 28; and severe group score = 42.

Research limitations/implications

Investigation of cut-off point by practitioners is vital to compare whether it aligns with the point determined by doctors in game addiction diagnosis. Future research should select critical item in order to reduce the number of questions and construct validity should be examined using confirmatory factor analysis.

Originality/value

This paper provides a comprehensive insight regarding severity of game addiction based on related criteria. As a result, treatment appropriate for each type of severity could be enhanced.

Citation

Wanpen, W., Itsarapong, P., Gornnum, S. and Yunibhand, J. (2022), "Development of a Thai gaming disorder scale for children and adolescents", Journal of Health Research, Vol. 36 No. 6, pp. 1047-1057. https://doi.org/10.1108/JHR-10-2020-0462

Introduction

Game addiction is now recognized as a major problem leading to a condition known as gaming disorder, a psychological condition caused by gaming addiction. In 2018, gaming disorder or gaming addiction was included in the 11th revision of the International Classification of Diseases (ICD-11) by the World Health Organization under the chapter on mental disorder in the section of substance addiction and behavioral addiction and highlighted the common brain activities during gaming disorder and substance use disorder [1]. There are approximately 2.2 billion gamers around the world, 3–4 percent, or an estimated 10 million of whom are game addicts. This figure is expected to rise in the future with children and teens being the most vulnerable to game addiction [2]. Based on the result of monitoring game addiction by the Child and Adolescent Mental Health Rajanagarindra Institute during the past five years, between the fiscal years 2014–2018, game addiction cases, as defined by the ICD-11, among children and teens (below 18 years) were seven times higher than before. The prevalence of game addiction problems among fourth to sixth graders was at 15% according to research in Thailand on the prevalence of gaming addiction. This indicated that Thai children and teens aged 6–18 years are most vulnerable to gaming addiction [3].

Gaming addicts show a persistent and recurrent pattern of gaming behavior. The Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5) notes that a gaming addict must manifest at least five conditions out of the following nine criteria: (1) preoccupied with gaming until it becomes the main routine in daily life (2) display of anxiety, irritation, or grief upon quitting games (3) continuous desire to extend gaming period (4) unable to reduce or control gaming period (5) neglecting other activities including daily routines, hobbies or interests (6) developing psychological and social problems by being overly engaged in gaming (7) deceiving family members or other people about gaming period (8) using games as a distraction or alleviation from negative moods (9) risk losing relationships, jobs and opportunities in education and career due to gaming [4].

Although various factors contribute to game addiction, the primary causes include child-related factors such as game exposure, length of time and frequency spent in gaming, source of access to games, personality and psychiatric disease. Other causes include family-related factors such as treatment toward children who spend an excessive amount of time on gaming, activities within the family and friends and environmental-related factors such as being surrounded by gamers, living near a game cafe, etc. Features and designs of games were also found to trigger game addiction [5]. Game addiction among children and teens brings about negative effects on various dimensions [6]. Furthermore, excessive game playing is associated with aggression in perception, interpretation, decision and reaction to situations [7].

Assessing the game addiction behavior is not only an indispensable process but also a baseline data prior to any treatment planning. Currently, foreign gaming addiction assessments comply with the psychiatric disease diagnosis (DSM-IV, V). Such assessments include the Game Addiction Scale for Adolescents (GASA) [8] and include seven indicators and 21 items. Cronbach's alpha was 0.94, The Internet Gaming Disorder Test (IGD-20 Test) [9] with 6 indicators of which the Cronbach's alpha was 0.88, The Internet gaming disorder scale (IGDS) [10] with 9 indicators and 27 items assessed polytomous and dichotomous had a Cronbach's alpha of 0.94 and 0.93 . And in the Game Addiction Screening Test (GAST) [11], a common game addiction scale in Thailand which includes the version wherein parents assess their children, the reliability was 0.94, and in the test where the children assess themselves, the reliability was 0.92 in order to optimize the classification of children and adolescents with a gaming disorder. However, since most were self-reporting their screening test, some answers may be invalid as respondents may not have answered truthfully, which is also a limitation found in previous research [9]. It is also subject to distortions as a thorough diagnosis from a psychiatrist or a medical staff member is necessary for determining whether a child is experiencing a gaming disorder [11]. However, research by Pornnoppadol et al. [11] has indicated that there is a lack of a standardized tool to validate diagnoses.

This study aimed to develop a Thai gaming disorder scale (T-GDS) for medical staff to test children and adolescents that complies with the diagnostic criteria; tests the reliability and validity and defines the cut-off point for categorizing the severity of game addiction into three levels.

Methodology

Participants

Data were collected during April–July 2020 at the Child and Adolescent Mental Health Rajanagarindra Institute. A total of 217 eligible individuals qualified to participate in the research. The inclusion criteria were as follows: (1) children and adolescents diagnosed with gaming disorder by a child and adolescent psychologist, (2) able to speak, listen, read and write Thai and (3) having been permitted by parents to participate.

Measurements

The participants were assessed with two assessment tools:

1. Children's Global Assessment Scale (CGAS) [12] was used as a gold standard tool. The most recent diagnosis from the psychiatrist was required and had to align with the game addiction diagnosis manual in DSM-5 of which the score range is 1–100. The group that did not exhibit any gaming disorder was expected to achieve high assessment scores whereas the group with a gaming disorder was expected to achieve lowers scores depending on symptom and function.

2. The T-GDS by Child and Adolescent Mental Health Rajanagarindra Institute based on the IGD-20 Test [8] was adapted from APA's Diagnostic and DSM-5 as indication criteria for game addiction assessment. It consisted of 6 criteria and 20 questions. The sample group responded to an interview by the medical staff with an introductory question being “Has your gaming pattern during the past 12 months signified any of the following ideas/emotions/behavior?” All items were rated on a scale: 0 (“not at all”), 1 (“perhaps no”), 2 (“perhaps yes”) and 3 (“absolutely yes”).

Procedure

This was a research and development study that followed the best practices for developing and validating scales for social science and public health science [13, 14]. The procedure consisted of seven steps.

1. Identifying topic, framework, objective, the scope of developing T-GDS in children and adolescents through planning with the working group.

2. Reviewing domestic and international literature related to game addiction in children and adolescents and concepts associated with developing psychological or medical assessment.

3. Designing questions and testing consistency in terms of content with the following details:

IGD-20 Test [8], approved by its creator, was selected as a reference for the development of T-GDS. In an attempt to create a draft of T-GDS in children and adolescents, the IGD-20 was translated and partly adapted. The scale comprised of 6 criteria and 20 indications.

To assess content validity, a draft of T-GDS was approved by three mental health experts who were also professors of child and adolescent psychiatry with over 10 years of experience in the field. The content validity was conducted through the investigation of the Index of Item Objective Consistency (IOC). After two revisions following the advice of the experts, all questions were in accordance with the criteria (IOC > 0.5) and were most appropriate for the Thai context.

Upon completion of the second revised version, three medical staff- a nurse, a psychologist and a social worker were asked to conduct an assessment on 15 children and adolescents replicating the actual sample group (1 medical staff for 5 children). The sample group was informed of the research objectives and procedures prior to agreeing to participate. Then the medical staff would question the children and adolescents to ensure that they understood the questions thoroughly and to minimize chances of any errors by the respondents.

1. (4)

   Determining sample group size and data collection method.

Purposive sampling was used and calculations by Buderer [15] were employed for sensitivity and specificity to obtain the sample group of game addicts and the proportion of non-addicts. The variables were as follows

• Sensi = proportion of the addict.

• Spec = proportion of the non-addict.

• P = prevalence of gaming disorder.

• Zα/22 = confidence interval 95%.

• d = error.

• N1 = sample size of the addict

• N2 = sample size of the non-addict

• N = sample size

  • Sample group being children and adolescents with gaming addiction.

Calculation formula TP+FN=[Zα/22(Sensi)(1−Sensi)]d2=56.6784

Solution N1=TP+FNP

Value substitution=56.67840.30=163.2

Thus, the size of sample with disorder was 163 people.

• Sample group of children and adolescents without gaming disorder

Calculation formula FP+TN=[Zα/22(Spec)(1−Spec)]d2

Value substitution=[(1.92)2(0.90)(0.10)](0.10)2=34.56

Solution N2=FP+TN1−P

Value substitution=34.560.70=49.37

The sample size of the non-gaming disorder was therefore 49.

The sample group of gaming addicts consisted of 163 individuals who were classified into four categories depending on the severity of the addiction. In total, 57 were grouped into the slight addiction category, 57 into the mild addiction category, 49 into the severe addiction and 49 into the non-addict categories respectively, making up the sample group of 212. Following the classification, the researcher selected cases through the purposive sampling method to match the size of each severity group using CGAS. The sample group was informed of the research objectives and procedures prior to deciding to participate in the research. In total, 217 children and adolescents were eligible to participate. In total, 86 of them (CGAS = 61–80 points) were classified with a mild gaming disorder, 58 (CGAS = 31–60 points) with a moderate gaming disorder, 20 (CGAS ≤ 30 points) with a severe gaming disorder and 53 (CGAS ≥ 80 points) in the other psychiatric disorder or non-addict categories.

1. (5)

   Experimenting with the scale on the sample group.

Two medical staff were trained on how to utilize the T-GDS. Assessment by the medical staff and the psychiatrist was performed on different occasions. Thus, the medical staff would not be informed of the CGAS result by the psychiatrist which had been conducted earlier. Prior to signing the consent form, the sampling group had been informed of the research objectives, procedures, period of time, benefits, risks and solutions by the researcher. Both assessors took turns in conducting an oral interview with the children. The scoring would be done on two separate assessment forms without any discussions. The interview lasted approximately 20−30 minutes.

1. (6)

   Assessing the quality of tools through reliability, validity and cut-off point analysis:

Once the assessment forms had been returned, the researchers conducted an initial analysis using descriptive analysis. An inter-rater reliability (IRR) investigation was conducted on assessments by both medical staff to evaluate the consistency between the assessors; Pearson's Correlation was conducted on all items and criteria, and item discrimination value was compared between the score of each item and correspondence corrected item-total correlation (CITC). In order to investigate if the equivalence of latency was in line with that presumed by the researcher, EFA was used. Then Cronbach's alpha coefficient from extracted variables was tested. Criterion-related validity was evaluated through concurrent validity. The tools involved included T-GDS in children and adolescents developed by the researchers and CGAS which is a medical assessment in categorizing participants into four groups initially. Eventually, ROC was employed in identifying the cut-off point for each group to categorize the level of severity analyzed by SPSS version 23.

1. (7)

   Gathering analysis for discussion and conclusion.

Ethical issue

This research was approved by the Research Ethics Review Committee for Psychiatric Research Involving Humans by the Department of Mental Health as per document no. 011/2563.

Results

Descriptive data

In total, 217 samples were eligible for the research including 151 males (69.6%) and 66 females (30.4%). The mean age of the sample was 14.11 years. There were 96 secondary school students, accounting for the majority of the participants (44.2%); 57 high school students (26.3%); 31 primary school students (14.3%); 18 unschooled/unidentified (8.3%); 12 Voc. Cert. Accountancy students (5.5%) and 3 participants in non-formal education (1.4%).

The mean score of the sample group from CGAS was 64.77 (SD = 18.09). The lowest CGAS score was 21, while the highest CGAS score was 92. The mean scores from T-GDS by assessors 1 and 2 were 24.88 (SD = 13.83) and 24.70 (SD = 14.14), respectively, with the minimum scores of both being 0 and the maximum of both being 58 (Table 1).

The respondents were categorized into four groups based on the severity of game addiction as per CGAS, namely, (1) normal: CGAS ≥ 80 (24.42%), (2) mild addiction: CGAS = 61–80 (39.63%), (3) moderate addiction: CGAS = 31–60, (26.73%) and (4) severe addiction: CGAS < 31 (9.22%) (Table 2).

Inter-rater reliability

The total intraclass correlation coefficient (ICC) value by both assessors showed a shared resemblance of 0.89 with the reliability range at 95% = |0.85| to |0.91|. When comparing all 20 items, the minimum ICC score was 0.72 with the reliability range at 95% = |0.65| to |0.78| and the maximum ICC score was 0.87 with a reliability range at 95% = |0.83| to |0.90|.

Correlation

In terms of assessment quality analysis, the scores by both assessors were summed up on a single scale (n = 434) for overall analysis. Pearson's correlation coefficient analysis found a positive correlation for all questions between |0.15| and |0.90| (p < 0.01). Considering the initial criteria analysis, positive correlation was observed between |0.50| and |0.79| with the most related criteria being factor 1 and factor 2 (r = 0.79, p < 0.01) followed by factor 3 and factor 4 (r = 0.78, p < 0.01). The correlation coefficient of scores from each item was explored in CITC. The item discrimination value for 20 questions was rated between |0.36| and |0.78| with item 7 (gaming uplifts mood) presenting the lowest discrimination value (0.36). However, no items with discrimination values lower than 0.20 were observed [16]. Cronbach's alpha value for all 20 items was 0.95.

Exploratory factor analysis

Exploratory factor analysis (EFA) was done using maximum likelihood estimation for factor extraction. The varimax technique was adopted for principal axis factoring. The obtained result was then used to consider factor loading derived from axis rotation. Variables with maximum factor loading were included in the component. Since the criteria for inclusion involved the item having factor loading over 0.5, the result from the initial analysis Kaiser-Meyer-Olkin (KMO) was 0.921, which was not lower than 0.5. Bartlett's test of Sphericity reported chi-square = 7549.89, df = 190, p = 0.000 implying the interrelationship of the factors studied and the appropriateness of the information for factor analysis.

Table 3. shows that the Eigenvalue of each of the four components extracted from EFA was between |2.54| and |3.80| and the variation ranged from |12.69|% to |18.98|%. The percent variance was 67.68. However, it must be noted that when considering communalities, items 7 and 18 were found to have an Extraction value < 0.50 and the minimum factor loading value being 0.38 and 0.35 respectively. Therefore, it was decided that these items be eliminated. The components were ordered in accordance with appropriateness and practicality, compared to the previous components and renamed some parts as follows.

Internal consistency

Cronbach's alpha coefficient analysis result of each component pointed out that component (1) preoccupation was equivalent to 0.90; component (2) withdrawal was equivalent to 0.94; component (3) loss of control was equivalent to 0.94; component (4) functional impairment was equivalent to 0.92 and the Cronbach's alpha analysis of all 18 questions remained at 0.95 (Table 3).

Criterion-related validity

The test was composed of two parts. The first part emphasized concurrent validity using Pearson's correlation coefficient to measure the T-GDS (18 items) and the assessment by doctors (CGAS) which is a gold standard. The result was equivalent to −0.858 indicating statistical significance at 0.01.

The second part focused on criterion-related validity using receiver operating characteristic (ROC) Curve and varying cut-off point of medical assessment via Youden's index = (Sensitivity + Specificity)−1 [17] to categorize addiction severity into three levels: mild addiction, moderate addiction and severe addiction. When compared to the standard assessment by doctors, the analysis result from Table 4 shows that the highest Youden index for mild addiction was 0.832 with the T-GDS score at 13.5, 92.68% sensitivity and 90.57% specificity. On the other hand, the highest Youden index for moderate addiction was 0.748 with the T-GDS at 27.5, 82.69% sensitivity and 92.09% specificity, whereas the highest Youden index for severe addiction was 0.837 with the T-GDS was at 41.5, with 85% sensitivity and 98.73% specificity (see Figure 1).

Discussion

The results of the study signify the reliability of T-GDS in children and adolescents since it had been developed from the IGD-20 Test [8] which aligns with criteria from DSM-5 from which adaptations and translations were made. Moreover, the research was systematically planned following best practices for developing and validating scales [13, 14]. The test which was adjusted following the expert advice twice in order to apply to the Thai context scored IOC > 0.50. The result was not only acceptable but also implied that there was a correlation between items and the assessment objective. As for intraclass correlation coefficient (ICC), a resemblance of 0.89 was observed [18] signifying shared assessment skills between the two assessors.

In terms of construct validity, the initial relationship between questions and six factors was investigated using Pearson's correlation coefficient. It was found that all the questions and indicators had a positive relationship with r from 0.15 to 0.90. Factor 1 (preoccupation) and factor 2 (tolerance) had a result as high as r is 0.79, p < 0.01. The overall Cronbach's alpha coefficient was 0.95 indicating a relatively high reliability. Prior to factor analysis, there was no indication of any item CITC lower than 0.20 [16] until EFA was run. Item no. 7 and no. 18 obtained the extraction and factor loading lower than 0.5 which could be due to the ambiguity and misleading aspect of the questions. Since these questions may have been misinterpreted, they were omitted and only 18 questions remained out of 20.

As a result of factor extraction, 4 out of 6 factors remained. Salience and preoccupation criteria and tolerance criteria were combined. According to DSM-5 [19], preoccupation is the condition wherein an individual is absolutely immersed in a game to an extent where it interferes with the process of thinking (obsession), feeling (craving) and behavior (over playing). These symptoms are closely related to those classified under tolerance criteria where an individual plays more frequently and intensely. In effect, the length of time children and adolescents are preoccupied with a game continues to increase [9, 20, 21]. As a result, behaviors under these two criteria are related and considered as one, namely “Preoccupation”.

Similarly, the previous two components relax/mood modification and withdrawal were combined as “Withdrawal”. Anxiety, irritation, or tantrums are triggered when an individual is prevented from playing a game to relax their mood and escape from reality, and in turn, manifests such undesirable behaviors. Thus, there is a correlation between these two assessment criteria as gaming has several purposes for children and adolescents – entertainment, acceptance and retreat from stressors. Inevitably, when prevented from playing games, they may express aggressive behaviors [20]. Moreover, preoccupation and withdrawal signify prioritizing gaming over other interests in daily life. This in turn is in accordance with “increased priority given to gaming” [22].

However, other components that were not combined to any factor were Loss of control and Functional impairment. “Loss of control” refers to the stage where children and adolescents cannot resist gaming despite their efforts to stop. This is related to “impaired control over gaming” [22] where gamers are unable to control aspects related to gaming such as beginning to play, frequency, intensity, length of time, types of game. “Functional impairment” may be defined as the effect of the game on responsibilities or personal relationships'. This is the stage where a gamer continues to play games despite their negative consequences, or, “continuation despite negative consequences” as mentioned in ICD-11 [22].

Furthermore, criterion-related validity also found that T-GDS (18 item) had a strong negative correlation with CGAS [23] because the scoring for CGAS opposed with that of T-GDS although both relied on the same gaming addiction symptoms and behaviors. In terms of test score interpretation for T-GDS (18 item), receiver operating characteristic (ROC) curve and cut-off score using Youden's index to generate a three-level severity scale were analyzed. It can be concluded that those scoring 0–13 points in the GDS in children and adolescents were non-addicts, 14–27 were mild addicts, 28–41 were moderate addicts, and ≥42 were severe addicts.

It is recommended the potential critical items with regards to game addiction assessment are further investigated in future studies. This not only reduces the number of assessment questions but also increases the accuracy of the assessment. However, since the number of severe game addicts is relatively small compared to other groups, ROC analysis and the cut-off score could be affected. Other limitations included the age of the sample group which is 8–18 years and inaccessibility to sample groups from other institutions. Therefore, it is recommended that future studies take into account construct validity through confirmatory factor analysis and evaluate the reliability of tools among a wider range of the child and adolescent population.

Conclusion

This study has shown the severity of the game addiction scale in children and adolescents to serve medical staff. The assessment consisted of 18 items under 4 diagnostic criteria. Each item was scored on a 4-point scale. The analysis points toward a good construct validity score with Cronbach's alpha equal to 0.95 and is suitable for assessing the severity of game addiction on a mild, moderate and severe basis. This assessment not only minimizes inaccuracies arising from self-questionnaires but also provides a powerful tool for appropriate treatment of game addiction in the future.

Conflict of Interest: None

Figures

Figure 1

Cut-off point graph for mild (left), moderate (middle) and severe (right) game addiction

Table 1

Mean score, SD, maximum-minimum scores from CGAS and T-GDS

Variables	Rater	Mean	SD	Min	Max
CGAS assessment score	Psychiatrist	64.77	18.09	21	92
T-GDS assessment score	Medical staff 1	24.88	13.83	0	58
	Medical staff 2	24.70	14.14

Table 2

Percentage as per game addiction severity level according to CGAS

CGAS groups	CGAS score	Sample group		n (%)
		Male (n = 151)	Female (n = 66)
Normal	≥81	31	22	53 (24.4)
Mild	61–80	66	20	86 (39.6)
Moderate	31–60	41	17	58 (26.7)
Severe	≤30	13	7	20 (9.2)

Table 3

Factor loadings of the items of T-GDS in children and adolescents and internal consistency

Initial factors	Items	EFA (α = 0.95)
		Loading	Component	α
Salience and preoccupation	Preoccupied with thoughts about previous game session	0.76	Preoccupation	0.90
	(2) Immersing self in thoughts about future game sessions when not playing a game	0.73
	(3) Playing games is a necessity	0.68
(2) Tolerance	(4) Time spent on gaming is continuously increasing	0.62
	(5) Spending money on higher levels in game	0.63
(3) Relax/mood modification	(6) Playing a game makes you easily frustrated	0.81	(2) Withdrawal	0.94
	(7) Play game to uplift your mood	0.38
	(8) Play game to distract self from frustration	0.66
(4) Withdrawal	(9) Feel upset when not playing game	0.77
	(10) Feel frustrated when not allowed to play game	0.84
	(11) Burst out in anger when unable to play game	0.76
(5) Loss of control	(12) Fail to quit playing game despite intention to do so	0.74	(3) Loss of control	0.94
	(13) Have never thought of quitting playing game	0.80
	(14) Fail to reduce the amount of time spent on playing game despite intention to do so	0.81
(6) Functional impairment	(15) Cannot admit to family the actual amount of time spent on playing game	0.64	(4) Functional impairment	0.92
	(16) Game playing has impaired relationship with your family	0.78
	(17) Neglect daily routines (chores, assignments, studies, etc.) as a result of game playing	0.81
	(18) Continue playing game despite its negative effects on daily life	0.35
	(19) Deprived of sleep as a result of game playing	0.67
	(20) Neglect engaging in favorite pastimes (watching movies, listening to music, playing sports) as a result of game playing	0.73

Table 4

Sensitivity, specificity, Youden's index and cut point of T-GDS (18 items) in the severity of game addiction scale

Game addiction level	T-GDS score	Sensitivity (%)	Specificity (%)	Youden's index	Cut point	Score range
Normal	0.0	–	–	–	–	0–14
Mild	≥13.5	92.68	90.57	0.83	14	14–27
Moderate	≥27.5	82.69	92.09	0.75	28	28–41
Severe	≥41.5	85.00	98.73	0.84	42	42–54