Quantitative analysis of the relationship between expressing gratitude and forgiveness and user sentiment on social media

Abstract

Purpose

The purpose of this study is to investigate how expressing gratitude and forgiveness on social media platforms relates to the overall sentiment of users, aiming to understand the impact of these expressions on social media interactions and individual well-being.

Design/methodology/approach

The hypothesis posits that users who frequently express gratitude or forgiveness will exhibit more positive sentiment in all posts during the observed period, compared to those who express these emotions less often. To test the hypothesis, sentiment analysis and statistical inference will be used. Additionally, topic modelling algorithms will be used to identify and assess the correlation between expressing gratitude and forgiveness and various topics.

Findings

This research paper explores the relationship between expressing gratitude and forgiveness in X (formerly known as Twitter) posts and the overall sentiment of user posts. The findings suggest correlations between expressing these emotions and the overall tone of social media content. The findings of this study can inform future research on how expressing gratitude and forgiveness can affect online sentiment and communication.

Originality/value

The authors have demonstrated that social media users who frequently express gratitude or forgiveness over an extended period of time exhibit a more positive sentiment compared to those who express these emotions less. Additionally, the authors observed that BERTopic modelling analysis performs better than latent dirichlet allocation and Top2Vec modelling analyses when analysing short messages from social media. This research, through the application of innovative techniques and the confirmation of previous theoretical findings, paves the way for further studies in the fields of positive psychology and machine learning.

Citation

Hitl, M., Greb, N. and Bagić Babac, M. (2024), "Quantitative analysis of the relationship between expressing gratitude and forgiveness and user sentiment on social media", Global Knowledge, Memory and Communication, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/GKMC-03-2024-0138

1. Introduction

Social scientists have discovered that every person can be described with 24 different character strengths that make up our personality (Peterson and Seligman, 2004). Every person is unique, and everyone possess these character strengths in different proportions. There are six broad virtue categories (Peterson and Seligman, 2004) in which these 24 characters strength falls under. This article will focus on two specific character strengths, and they are forgiveness and gratitude.

Forgiveness and gratitude are character strengths that relate to positive psychological processes and well-being. The goal of positive psychology is to shift the focus of psychology from solely addressing and fixing negative aspects of life to also nurturing and developing positive qualities. Some studies have shown that people who reported greater levels of forgiveness and gratitude also reported less anger, subjective feelings of loneliness and fewer depressive symptoms (Breen et al., 2010). These two feelings are important and useful to monitor when observing people’s behaviour in the context of bullying. Some findings validate that when individuals activate moral-self schemas like gratitude and forgiveness, they develop a predisposition to choose behaviours that are suitable for the given situation in the context of bullying (García-Vázquez et al., 2020). They have also found that regardless of gender or age, forgiveness, gratitude and happiness are all valuable in those situations.

Forgiveness is a feeling that can be expressed of oneself or of others. It is good to recognise that because forgiveness of oneself appears to have the strongest connection to well-being and it has the most dramatic effects on people who have relationship issues (Toussaint and Friedman, 2008). For forgiveness seeking revenge and avoiding transgressor are negatively correlated with life satisfaction and this can be explained through experience of grudge (Szcześniak and Soares, 2011). Forgiveness is not stable through time and sometimes individuals would not forgive something that would they forgive some time earlier (Sapmaz et al., 2016) so it is necessary that it is observed through longer period because forgiveness can variate and so can its consequences.

Gratitude depends on certain domains of human life such as work, health and relationships. It can variate in different cultures and within different people with different kinds of human trait such as neuroticism and extraversion. Studies that covered all these different cases have found that people who reported higher levels of gratitude also reported higher levels of satisfaction in each domain of life (Robustelli and Whisman, 2018). Some countries such as Japan, have tendencies to show less positive and negative emotions so they would have lower ratings in comparison with for example, USA but it does not change the fact that gratitude still correlates with satisfaction of life even for those countries.

The reason why we will observe only forgiving and grateful people is because these two positive emotions have important connection with well-being that has previously been studied (Toussaint and Friedman, 2008). Additionally, some studies found out that gratitude explained a significant amount of variance of total tendency to forgive (Félix, 2007). Both forgiveness and gratitude contribute to well-being. To conclude, these findings were additionally confirmed for gratitude in two recent meta-analyses conducted by Diniz et al (2023) and Kirca et al (2023) while the book “Forgiveness and Health: Scientific Evidence and Theories Relating Forgiveness to Better Health” published by Toussaint et al. (2015) in 2015 gives numerous evidence of the importance of forgiveness for overall well-being.

The first type of analysis conducted in this research is sentiment analysis of posts (Bagić Babac, 2023). Through this analysis, a numerical value will be derived from text to indicate how positive or negative posts are. Two independent groups of users will be created: one group consisting of users known for expressing gratitude and forgiveness, and another group consisting of random users who express these fillings more rarely. Sentiment analysis will be performed using the R package Syuzhet. Users and their posts will be observed over a longer period (2 years) to minimise the impact of major events on the results. This research will aims to demonstrate that people who express more feelings of gratitude and forgiveness have a higher sentiment of all their posts during the observed period.

One way to determine whether a person is positive or negative is to conduct a study in which we ask participants questions, and they respond, and conclusions are drawn based on their answers (Wade and Worthington, 2005). A better option, which allows for a much larger number of samples for examination and the ability to draw better conclusions, is to use data from social media. A good social network that allows us access to large amounts of text for various analyses is X. Its key feature is that it only allows short messages with a maximum length of 280 characters, which has both positive and negative aspects. The advantage is that the large amount of data can be gathered and performing analysis can be relatively quickly done because the text is concise. However, the downside is that the text is poorly grammatically structured, it is difficult to understand slang, lack of understanding the complete context in texts, and the need to primarily consider only English texts because good lexicons for sentiment analysis do not exist for most other languages (Martínez-Cámara et al., 2014). Texts also need to be well-filtered to obtain better results. For example, the use of emojis in text can provide more precise results in sentiment analysis, but each platform has different emojis, and they would need to be interpreted first, which can pose a problem (Wunderlich and Memmert, 2020). However, if existing tools are used correctly, that is, if data is properly prepared for analysis, they can accurately classify up to 95% of texts on average (Alrumaih et al., 2020). It is also necessary to consider the contexts from which posts are collected and analysed. People from different countries will have different experiences and perspectives on things happening around them. Research that considers the country from which user texts are collected on social media has shown that users on certain topics such as COVID-19, hospitals, health, death and similar (Kausar et al., 2021), express different emotions.

The second type of analysis conducted is topic modelling analysis and with this the main topics from the text can be identified and later discussed. Using several topic modelling algorithms (Brzić et al., 2023), the most significant topics from both groups of users will be extracted. If the topics are well-defined, there will be an attempt to explain why they are distributed in such way and why they occur within specific groups. There will also be an attempt to show what people are grateful and forgiveful for and what are the differences in topic sentiments between these two groups, which is whether users who express the more forgiveness and gratitude speak with a different sentiment about the obtained topics or not.

Topic models are a great thing that helps us find the main topics in a larger and unstructured text, but they can be significantly impacted by the length of the documents being analysed. To elaborate, some researchers have found that a more effective model can be trained by combining shorter messages into longer text (Hong and Brian, 2010). Social scientists are sometimes not sure which algorithm they should choose when exploring social media data. There are several types of topic models. Each of them uses different learning approaches, some of them use generative statistical approach (latent dirichl et al. location (LDA)), some use linear algebra approach (NMF) and some use embedding approach (Top2Vec, BERTopic). Research that has dealt with the comparison of different algorithms have conclude that using BERTopic to extract valuable information from brief and unstructured text exhibits the greatest promise in terms of embedding-based topic modelling (Egger and Joanne, 2022). As time progresses, the subjects within a collection of documents undergo changes. Failing to account for time when modelling topics can lead to difficulties. Therefore, it would be useful to incorporate topic evolution to enable people to recognise topics within their context and observe the changes that occur over time (Alghamdi and Alfalqi, 2015). We won’t take time into account because it is challenging thing to do and because our data won’t cover extremely large period of time. Also, one of the problems that occurs in some topic modelling algorithm (for example LDA) is too much vocabulary in the text, and consequently it is difficult to create a good model that will be predictive and interpretable. Usually, the process involves eliminating the most and least commonly used words. However, in the case of large collections, this pruning procedure may eliminate significant terms and restrict the range of the models (Dieng et al., 2020).

2. Methodology

2.1 Hypothesis

The hypothesis of this study is that users who frequently post expressions of gratitude or forgiveness will exhibit a more positive overall sentiment in their posts during the observed period compared to users who post such expressions less frequently. This hypothesis will be tested by creating two independent groups. Group A will consist of users who frequently express feelings of gratitude or forgiveness, while Group B will consist of random users. After identifying these users, we will collect all their posts over a extended period and analyse sentiment of their posts. The reason for selecting a longer period is because sentiment varies over time due to external events (Bagić Babac and Podobnik, 2023), which can be positive, negative or neutral. The average sentiment of each group will be calculated, with the expectation that Group A will have a significantly higher average sentiment than Group B. This is based on the idea that expressing gratitude and forgiveness helps individuals release negative energy, therefore focusing more on positive aspects of life and improving their well-being (Sapmaz et al., 2016). Different variables from both groups will also be compared to determine if there are additional differences between the groups. For topic modelling, it is expected that Group A will have more positive topics than Group B. Even if the topics are similar expectation is that the topics in Group A will have higher average sentiment then the topics in Group B.

2.2 Collecting data

Data were collected from X using an application called Facepager (Jünger and Keyling, 2019). To use Facepager, a Twitter Developer account must first be created, to access the necessary information from X. For Group A, posts need to be using keywords related to gratitude or forgiveness (example of query is: forgive OR forgiveness OR gratitude OR grateful). After collecting user data, the next step is to download all posts from these users within a specific period which is from May 2021 to May 2023. For Group B, a different query with neutral keywords will be used, to gather data from random users. Examples of neutral keywords in the English include that, have, with, there, they, from and many others.

2.3 Cleaning data

The programming language used for data cleaning is R, and the development environment used was RStudio. Collected data were organised into four tables: two for the users of Groups A and B, and two for their corresponding posts. One important column in the user’s table is username. This is crucial because Group A and B must be independent. With this value, duplicates users collected during scraping could be removed. Table 1 provides examples of user data, and Table 2 examples of posts. All duplicates and non-English posts were excluded from the analysis. Duplicate posts were removed because some users repeated certain posts multiple times, which would introduce spam and unnecessary noise into the data. Mentions starting with ‘@’ as well as links, hashtags and special characters (such as emojis) were removed from the posts to achieve more accurate sentiment analysis. A new column indicating the average number of posts per day for each user was added to the user tables. This value was obtained by grouping the posts by user ID, calculating the row count for each group and appending this count to the corresponding user ID in the user tables. Because sentiment varies over time, users who posted on fewer than 15 different days were removed from data sets. Someone who posted 15 posts in one day does not cover same period as someone who posted 15 posts in 15 different days.

Figures 1–4 show density plots of the variables followers, following, post count and number of posts per day for the user tables of Groups A and B. Group A is represented by blue colour and Group B by red colour. From the graphs, it can be seen that users in Group A have more followers but, in contrast, follow fewer people. They have fewer posts overall and fewer posts per day over the observed two-year period. Users who had more than 4.5 posts per day were excluded from each group because we wanted to obtain users with a more uniform distribution of posts. In Figure 5, the length of all posts for Groups A and B is shown. The graph indicates that posts in Group A are longer than posts in Group B. To ensure that all these results are accurate and not obtained by chance, we will conduct statistical testing on these variables in the results section to confirm the significance of these group differences.

2.4 Sentiment analysis

Sentiment analysis is natural language processing (NLP) technique used to determine the emotions present in a body of a text and measure its positivity or negativity. For this study, the R package called Syuzhet was used to conduct sentiment analysis. This package performs unsupervised sentiment analysis by using pre-existing dictionaries to return sentiment values (Bagić Babac, 2024). Syuzhet consists of four different sentiment lexicons (Syuzhet, Affin, Bing and NRC) which provide sentiment values for the text. The primary method used from this package is get_sentiment(), which takes two arguments: a vector of words from the text and the name of the desired lexicon. The Syuzhet lexicon contains the highest number of positive and negative words, so this lexicon will be mostly used in the further analysis. The average sentiment value for each user was calculated by grouping posts by their parent ID, calling get_sentiment() method for each post and then computing the average sentiment for each user. This average sentiment was appended as a new column in the user data set.

2.5 Topic modelling

Topic modelling is a statistical modelling technique, and it is a type of unsupervised machine learning. It helps us to discover hidden topics within a collection of documents. We will use topic modelling because it is almost impossible for humans to manually identify all the main topics in a large data set. The goal of topic modelling is to find best meaningful topics and explain why they occur in certain proportions. There are many topic modelling algorithms, some well-established LDA and latent semantic analysis) and others more recent, that have increasing attention from scientist and researchers in recent years (BERTopic, Top2Vec, Non-Negative Matrix Factorisation and Correlation Explanation topic model). This research presents results from three algorithms: LDA, BERTopic and Top2Vec.

2.6 Latent dirichlet allocation

LDA topic analysis was performed in R using the topicmodels package. LDA is generative probabilistic model built as a three-level hierarchical Bayesian model, consisting of topics, documents and words (Blei et al., 2003). It uses two separate distributions (document-topic and topic-word distribution), and it iteratively estimates the probability of each word belonging to each topic. The goal is to find the most likely topic for each word within the entire topic structure. The text was cleaned to achieve the best possible results from the model. Capital letter, special characters, punctuations and stop words were removed. White spaces were stripped (multiple whitespaces are collapsed into one), and each word was stemmed using Porter’s stemming algorithm. Stemming is a NLP technique that reduces inflected words to their root form.

2.7 BERTopic and Top2Vec

These two topic modelling algorithms share similarities in their implementation, both using an embedding approach, which retains the original text structure for their transformer models. BERTopic (Bidirectional Encoder Representations from Transformers) uses pre-trained transformer-based language models to generate document embeddings. These embeddings are then clustered, and topic representations are generated using the class-based term frequency-inverse document frequency (TF-IDF) procedure. TF-IDF is a numerical statistic that indicates the importance of a term in document within a collection of documents. BERTopic can generate coherent topics and performs well compared to both traditional topic modelling models and newer clustering-based approaches in various benchmark evaluations (Grootendorst, 2022). BERTopic analysis was conducted in Python using the bertopic package.

Top2Vec is a novel approach that uses combined document and word semantic embeddings to identify topic vectors. Unlike traditional methods, Top2Vec does not rely on stop-word lists, stemming or lemmatisation and it automatically determines the optimal number of topics similar as BERTopic. The resulting topic vectors are embedded together with document and word vectors, and the distance between them reflects their semantic similarity. Experimental results indicate that Top2Vec identifies highly informative and representative topics that outperform probabilistic generative models trained on the same corpus (Angelov, 2020). Top2Vec topic analysis was also performed in Python using the top2vec package.

3. Results

To successfully test the hypothesis of this study, a comparison between Groups A and B needs to be conducted to gain a clearer understanding of user’s characteristics. Users in Group A were randomly collected using search keywords related to forgiveness and gratitude, while Group B consists of random X users. Data were collected for 1,191 users in Group A and 1,072 users in Group B. The number of posts for Group A is 283,393, while for Group B, it is 218,780. Although this is not a exceptionally large number of users, enough posts were collected to obtain meaningful results in sentiment and topic modelling analysis.

The independent samples t-test is used to compare the characteristics of the two groups of data. User data were randomly sampled by searching for users with specific keywords in their posts, ensuring no bias in selecting users. We achieved independence by removing duplicate users so that both groups contain only unique users. Normality is assumed without testing because, according to the Central Limit Theorem, the distribution of the sample mean tends to be approximately normal when we have large samples (n > 30) (Walpole et al., 1993). Table 3 shows results of testing the similarity between the variances of certain variables from data set using the F-test. From these results, we see that only Number of posts per day assume equal variances due to the high p-value. p-value higher than 0.05 indicates lack of evidence to suggest different variances. Other variables show different variances.

Knowing if variances are equal, we can perform the t-test on these variables to see if their mean differences are statistically significant. Table 4 shows the results of specific variables in the data sets, including their t-values, p-value, degrees of freedom (number of values tested from both groups) and mean. For: number of people following, number of posts, number of posts per day per user and average length of posts, their p-values are less than 0.05. Therefore, we reject the null hypothesis and accept the alternative hypothesis, indicating significant differences between these variables in the two groups. Users from Group A, on average, follow fewer people, publish fewer posts per day in observed time period and have fewer posts overall, but they write longer posts than random users. For the number of followers, we cannot reject the null hypothesis because the p-value is 0.1818, indicating that both groups have the same number of followers on average. The finding that grateful and forgiving people follow fewer accounts and post less frequently suggests they may engage in more meaningful interactions rather than seeking quantity in their social media presence. Writing longer posts may indicate that they take more time to express their thoughts and emotions thoroughly, potentially leading to higher quality content.

It is also important to check whether the proportion of posts containing the words gratitude and forgiveness differs significantly between the two groups. This should be a fundamental difference, as Group A was formed by searching for users who express these feelings more frequently. An independent two-sample t-test found that the difference in the percentage of posts containing words related to forgiveness and gratitude between Group A (mean = 0.0189; standard deviation = 0.1362) and Group B (mean = 0.00287, standard deviation = 0.05349) was significant (t = 52.081; p < 2.2e-16). These results allow us to say with high certainty that the two groups differ in the number of posts expressing gratitude and forgiveness, enabling a meaningful sentiment analysis on these groups.

For sentiment analysis, it is beneficial to have an even distribution of posts over time to avoid the influence of specific events that could significantly impact the results of sentiment analysis. Similarly, for topic modelling analysis, it is preferable to avoid a dominance of posts from specific months, as this would result in the overrepresentation of a single major event. The initial data set collected was much larger than the current one, but users and their number of posts were filtered to achieve a better distribution and avoid testing users with a small number of posts. The distribution of collected posts over the period from May 2021 to May 2023 for Groups A and B is shown in Figure 6. Group A is represented by coral colour and Group B by turquoise colour. Their distributions are similar, but Group A has slightly more posts each month. This distribution is not ideal but there were certain limitations when we scraped this data. This problem is explained in more details in later chapter Conclusion.

3.1 a. Sentiment analysis of posts using R package Syuzhet

Sentiment analysis of the text was performed using 4 different methods (Syuzhet, Affin, Bing and NRC) offered by the Syuzhet package. The results of all these methods are shown in Table 5. Each method uses different scales for evaluating the sentiments of individual words. Again, the t-test of independent samples will be used to check whether the differences in average sentiment values are significant. What can be immediately noticed is that the calculated average sentiments of all methods of Group A are almost twice as high as the average sentiments of Group B. For the Syuzhet lexicon, the difference in sentiment between Group A (mean = 0.6551; standard deviation = 1.3218) and Group B (mean = 0.3573; standard deviation = 1.1000) was significant (t = 85.069; p < 2.2e-16). These results confirm that the sentiment difference between Groups A and B is statistically significant. Therefore, we can conclude that people who express more gratitude and forgiveness have a more positive sentiment in all their posts during the observed period compared to users who express these feelings less frequently. The higher positivity in their posts aligns with the theory that expressing gratitude and forgiveness enhances emotional well-being (Breen et al., 2010). This positive sentiment can be a marker of their overall mental health and satisfaction.

In all further comparisons of sentiment, the Syuzhet method will be used because its lexicon contains the highest number of words (10,748) if we exclude repeated words from the NRC lexicon (Kim, 2022). Figure 7 shows the average sentiments of Groups A and B from May 2021 to May 2023. Group A is represented by coral colour and Group B by turquoise colour. From the graph, we can see a significant difference between the average sentiment of these groups. It can also be observed that the average sentiments of each group do not vary much over time but show a slight decline. This decline is because, as we get closer to dates nearer to the start of post scraping, there are more collected posts in those months. For a larger number of posts, the overall sentiment tends to be lower. The correlation between the average user sentiment per month and the number of collected posts per month is −0.79 for Group A and −0.73 for Group B, indicating a high negative correlation in both groups. This means that the more posts we have in a given month, the lower the average sentiment for that month will be.

3.2 b. Results of topic modelling algorithms latent dirichlet allocation, Top2Vec and BERTopic

3.2.1 i. Latent dirichlet allocation.

LDA did not produce good results because the number of topics that the algorithm must generate depends on the user’s preference, making it challenging to guess the ideal number for the input data set. The advantages of this algorithm are that it does not require knowledge of the domain of the input data set, and the number of topics generated is much smaller than those of word-embedding algorithms, making it easier to interpret the created topics. To use the algorithm, the text first had to be modified. Punctuation marks were removed, all uppercase letters were converted to lowercase, stopwords were removed, all excess spaces were removed and all words were converted to their root form (stemming). Empty lines that appeared due to text modification were also removed. Finally, the variable indicating the number of topics that the algorithm would generate was set to 100. If a larger number of topics were used, the same words would start to appear in different topics, indicating that the algorithm was generating too many topics. Results with different numbers of generated topics were analysed, but it was difficult to determine which was the best because how good the results are, is depending on subjective assessment. The results of the topic analysis for Group A are shown in Figure 8. Among the 10 most common topics in Group A there are topics related to religion, food, sports, politics, celebration, technology and others.

However, for Group B, the results were not as good. Figure 9 shows the results of that analysis. In the 10 most common topics of Group B, it is hard to determine what certain topics represent because these topics contain words that do not carry much information. Most topics were too specific and not homogeneous (several topics were found in one topic). Therefore, we will not explain the results of the LDA topic modelling analysis in depth, as it will not add value to this analysis.

3.2.2 ii. Top2Vec.

The next algorithm used for this analysis is Top2Vec. The biggest advantage of this algorithm is its ability automatically detect topics from the text using pre-trained word vectors, making it easy to create documents and meaningful embedded topics. Compared to the LDA algorithm, Top2Vec works well with large data sets, can automatically find the ideal number of topics of the input text, and does not require any pre-processing. However, it can generate a lot of outliers and numerous topics, making them difficult to interpret.

For Top2Vec topic analysis, a pre-trained sentence transformer model called the Universal Sentence Encoder was used. This model optimised for text longer than a single word. It is designed to capture the semantic meaning and context of sentences. As the size of the sentence increases, computation and memory usage increase quadratically (Cer et al., 2018), making this model impractical for longer texts. It is best for text not longer than a single sentence, a short paragraph or a phrase, which is ideal for a data set consisting of posts limited to 280 characters. The ngram_vocab parameter was set to true to allow generated topics to include bigrams (bigram is a two-word sequence of words, for example “he likes”).

The results for the Top2Vec algorithm are shown in Table 6. In the 10 most frequent topics of both groups, there is one topic in Group A (topic 9) that contains completely random words that cannot be interpreted. Other topics lacking specific meaning include topics 6, 7 and 10 in Group A and topics 2, 3, 4 and 9 in Group B. Many of these topics consist of common insults found on social media, and they are unavoidable when collecting data from X. However, there are some discernible topics, such as topics 1 (success strategies), 2 (personal dignity), 3 (gratitude), 4 (Twitter), 5 (online entrepreneurship) and 8 (biblical books) in Group A and topics 1 (football), 5 (Twitter), 6 (songs), 7 (curiosity), 8 (war) and 10 (excitement) in Group B. These results suggest that the Universal Sentence Encoder may not perform well on a large data sets for text analysis on X, so another pre-trained embedding model for Top2Vec should be considered.

3.2.3 iii. BERTopic.

The final algorithm used for topic analysis is BERTopic. Its advantages and disadvantages are similar to those of the Top2Vec algorithm due to their similar implementation. For BERTopic, a pre-trained sentence transformer model called all-MiniLM-L6-v2 was used. This model is designed for short text, such as one sentence or a short paragraph, and generates contextualised word embeddings to capture the meaning and context of words in each sentence. The ngram_range parameter was set to be in the range from 1 to 3, allowing words to be grouped into unigrams, bigrams and trigrams.

The bertopic package provides functions to display the topics, their most frequent words, their corresponding frequencies and bar charts showing the 10 most frequent topics. Figures 10 and 11 show these displays for Groups A and B, respecitvely. BERTopic generated 3,691 topics for Group A and 3,053 topics for Group B. Outliers (generated documents that did not found their place in any topic) make up 36.23% of Group A and 34.05% of Group B. It is impractical to provide explanations for all generated topics due to their large number, so only the 10 most common topics will be explained. All topics in the 10 most represented are clearly meaningful, unlike those generated by the previous two algorithms.

In Group A, the most represented and positive topic is birthday, marked as topic 0. This reflects how people often wish happiness, health and love when congratulating birthdays, spreading positive energy. A neutral topic in Group A is Twitter, which is not surprising because the data was collected from X (formerly known as Twitter). Topic 6 discusses COVID-19, and it has been quite relevant in the recent years and it has brought a lot of negative things. As its influence is weakening, users no longer write negative comments, but they now write more neutrally about COVID-19. Topic 7 talks about the racial discrimination, a inevitable topic of conversation. It is the only negative topic from 10 most common in Group A. Other general topics include topics 2 (health care), 4 (human genders), 5 (books) and 8 (TV shows). Gratitude appears as topic 9, which is expected because users expressing this sentiment were specifically targeted and this is the most positive topic in Group A. Topic 3, which includes prayer, is interesting as studies suggest that people who pray more often are more willing to forgive to their loved ones (Lambert et al., 2010). This data does not provide strong evidence for correlation of forgiveness and prayer.

Group B should represent the most common topics on X over, as users were randomly selected, though the sample size is not large enough (1,072) to definitively determine the most represented topics on X. In Group B, the most represented topic is Twitter, similar to Group A. The most positive topic is birthdays (topic 4), because congratulating birthdays are frequent occurrences on social networks. General topics include topics 1 (education), 2 (photos), 3 (videos), 6 (human genders) and 7 (books). Negative topics related to war are topics 5 and 8, reflecting the start of the war in Ukraine during the data collection period. Topic 9 is also negative, containing a specific racial insult common on social media.

The values of sentiments calculated with Syuzhet method for each separate topic for Groups A and B are shown in Table 7. From these results, even for the same topics such as birthdays, Twitter and books, users in Group A tend to write slightly more positive messages than users in Group B. The average calculated sentiment for the 10 most common topics in Group A is 1.052, while for Group B it is 0.478. This significant difference in average sentiment supports the idea that individuals in Group A, who express more positive emotions tends to have higher overall positivity in their communication.

4. Discussion

4.1 Theoretical implications

This research has shown that positive sentiment is higher in individuals who express forgiveness and gratitude more frequently. This indicates that these two emotions contribute to more positive thoughts in individuals, resulting in more positive messages on social media. The findings support theories in positive psychology that suggest expressing gratitude and forgiveness contributes to positive emotional states, even though the study focuses on sentiment rather than direct measures of well-being.

From the results of the topic analysis, it can be concluded that people expressed the most gratitude towards doctors and health care, particularly during the COVID-19 pandemic. Forgiveness was predominantly expressed towards God, with people looking for forgiveness from God through prayer. One of the topics that also mentioned forgiveness, was the topic of student debts. The USA faces significant issues with high student debts towards universities (Board of Governors of the Federal Reserve System, 2023), leading to the introduction of debt forgiveness programs to mitigate student’s economic situations. The identified topics related to gratitude and forgiveness suggest a theoretical link between these emotions and the content people are likely to discuss, which can inform future research on the contextual triggers of these emotions. The successful application of the BERTopic algorithm to identify compact topics underscores the utility of advanced machine learning techniques in sentiment analysis, contributing to theoretical discussions on the coherence and interpretability of topic models. Result indicates that use of transformer-based topic modelling algorithms like BERTopic on social media data gives far better results than generative probabilistic topic modelling algorithms like LDA, as indicated by other works (Kastrati et al., 2023).

4.2 Practical implications

During the data collection process for users, we were unable to gather certain user details that would have been extremely useful for a more comprehensive research analysis. Some of these details are age, gender and location. Users are not required to provide these details on their profiles, so the majority of users did not have this information’s defined.

Parametric statistical tests, such as the t-test, require that the data follow a normal distribution. In this study, we assumed normality without formally verifying it. We made this assumption because we were working with groups of a large number of people. The tests we conducted could potentially gain additional significance if we not only assumed normality but also verified it either computationally (using the Kolmogorov–Smirnov test) or graphically (by creating a histogram or Q-Q plot).

In future research, better packages than Syuzhet could be used because Syuzhet does not take modifiers into account, which can significantly alter the meaning of a sentence. When collecting users, it may be advisable to focus on verified X accounts, as they are less likely to produce unstructured and poor-quality texts, and they would produce fewer spam messages.

The topics generated by the BERTopic algorithm appear to be relatively compact, meaning that it is clear which themes each topic contains. However, this study did not conduct an analysis on topic coherence to obtain an objective measure on how similar certain words in topics are, which could be addressed in future research to obtain more reliable and interpretable topics.

4.3 Limitations

There is a limitation that if someone wants to retrieve all the posts from a user excluding reposts (as reposts are not relevant because they are simply forwarded by the user and do not reflect their own views), only the latest 800 posts per user can be downloaded. The aim was to collect all posts from users over the past two years. However, because there was a limitation on the number of posts that could be scraped for each user (800), a problem occurred in a way that a larger number of posts were collected in the interval from May 2022 to May 2023, while a much smaller number of posts was collected in the interval from May 2021 to May 2022 as shown in Figure 6. Ideally, it would be desirable to have an equal number of collected posts for each month over the past two years, but this is challenging to achieve with such a limitation.

5. Conclusion

Sentiment analysis provided good results, confirming the hypothesis of the study. Future research could investigate how gratitude or forgiveness alone contributes to the sentiment of individuals on social media. It would also be interesting to see if there are differences among male and female users as well as among younger and older users, in expressing those two emotions. The results of this research could be used in the future for developing mental health interventions that encourage expressions of gratitude and forgiveness on social media to improve mental health and well-being. We could also use social media behaviour patterns to identify individuals who might benefit from these positive psychology interventions.

Forgiveness and gratitude play essential roles in individual lives, not only yielding positive consequences for individuals but also for the broader communities in which they live. Encouraging forgiveness and gratitude in various settings such as health care, schools and workplaces can contribute to a more compassionate society. Finally, through the application of innovative machine-learning techniques (Puh and Bagić Babac, 2023) and the confirmation of previous theoretical findings (Poje et al., 2024), our research paves the way for further studies in the field of positive psychology and machine learning.

Figures

Figure 1.

Density graph for number of followers for groups A and B

Figure 2.

Density graph for number of people following for groups A and B

Figure 3.

Density graph for number of posts for Groups A and B

Figure 4.

Density graph for average number of posts per day for groups A and B

Figure 5.

Density graph for number of words in posts for groups A and B

Figure 6.

Distribution of posts for groups A and B over the period from May 2021 to May 2023

Figure 7.

Average sentiments of groups A and B over the period from May 2021 to May 2023

Figure 8.

LDA topic modelling results for group A

Figure 9.

LDA topic modelling results for group B

Figure 10.

BERTopic topic modelling results for group A

Figure 11.

BERTopic topic modelling results for GROUP B

Table 1.

Example of user dataset

ID	Followers	Following	Posts count	Description	Username	Average number of posts per day
313	325	371	10,825	Text me	DSouthernLady	2.59
12113	6,386	206	14,311	What can we do today to express gods love to those around us?	sarahbowling	3.72
21555	130	58	3,759	Put your future in good hands	NasiraRazzaq	3.29

Source: Tables 1 and 2 that contain publicly available data – online on X (Twitter)

Table 2.

Example of posts dataset

ID	Parent ID	Created At	Posts
151004	1059	2023–05–10	Gorgeous love the orange lilies! beautiful colour
306481	2105	2023–05–17	The best investment you can make is in yourself Focus on personal growth and development, and the money will follow
173656	1207	2021–07–28	Love and care for our mother earth and it will take care of you more

Source: Tables 1–2 that contain publicly available data – online on X (Twitter)

Table 3.

F-Test of equality of variances for specific characteristics of Datasets A and B

	No. of followers	No. of people following	Posts count	No. of posts per day per user	Average length of posts
F-value	40.405	0.45876	0.63874	0.99323	1.1132
p-value	2.2e−16	2.2e−16	5.357e−14	0.9084	2.2e−16
Standard deviation – Group A	87438.7	1524.064	16176.35	0.9571	13.684
Standard deviation – Group B	13755.76	2250.148	20240.32	0.9604	12.969
95% confidence interval	(35.944, 45.403)	(0.408, 0.516)	(0.568, 0.718)	(0.884, 1.116)	(1.104, 1.122)

Source: Authors’ own work

Table 4.

Independent samples t-test for specific characteristics of Datasets A and B

	No. of followers	No. of people following	Posts count	No. of posts per day per user	Average length of posts
T-value	1.3359	−2.2113	−3.7143	−3.4158	43.839
Degrees of freedom	2,261	2,261	2,261	2,261	502,109
p-value	0.1818	0.02714	0.00021	0.00065	2.2e-16
Mean Group A	5891.285	726.0302	7723.69	2.3222	17.868
Mean Group B	2460.411	906.6744	10605.26	2.4600	16.210
95% confidence interval	(−1607.671, 8469.417)	(−340.86, −20.429)	(−4403.004, −1360.127)	(−0.217, −0.059)	(1.584, 1.732)

Source: Authors’ own work

Table 5.

Results of sentiment analysis with four different lexicons from Syuzhet package

Lexicon/method	Mean for Group A sentiment	Mean for Group B sentiment	Group AGroup B
Syuzhet	0.6551	0.3573	1.8335
Afinn	1.4042	0.7316	1.9194
Bing	0.4758	0.2448	1.9440
NRC	0.7269	0.3729	1.9491

Source: Authors’ own work

Table 6.

Top2Vec Topic modelling results for both Groups A and B

Topic no.	Group A	Group B
1	Perseverance, achieve success, achieving success, stay motivated, stay positive	Utd fans, premier league, man und, arsenal fan, utd
2	Self respect, stay humble, be honest, human dignity, aint nobody	Hes gonna, thinks hes, hes, he knows, he wants
3	Aw thank, thank yous, aww thank, incredibly grateful, woohoo congratulations	Fucking idiot, stfu, fuck off, fucking stupid, moron
4	Tweets, tweeting, tweeting about, tweet, Twitter feed	Fucking idiot, cmon man, cant spell, ah yes, fucking stupid
5	Affiliate marketing, digital marketing, an entrepreneur, start affiliate, launch webinar	Tweets, tweet bro, tweeting, tweeting about, tweet
6	Fuck wit, fuck off, fucking sucks, aint nobody, aint shit	New song, favorite song, fav song, whole album, song
7	Shit wtf, fucking weird, ah yes, was not expecting, fucking sucks	Why does, wonder why, why bother, wondered why, wondering why
8	Philippians esv, galatian esv, ephesians esv, colossians esv, ephesians esv	Ukraine war, Russian military, Russian invasion, Russian troops, Ukraine nazi
9	Comr, abeg, shamiri, popp, beyonc	Do not deserve, self-respect, stop giving, be honest, fuck em
10	Weeks ago, fucking sucks, was not expecting, have not already, shit wtf	Can not wait, cannot wait, super excited, yay yay, very excited

Source: Authors’ own work

Table 7.

Sentiment of 10 most common topics found with BERTopic for Group A and B

No. of topics	0	1	2	3	4	5	6	7	8	9
Group A	2.397	0.375	1.400	1.214	0.776	1.230	0.430	−0.772	0.897	2.575
Group B	0.316	0.945	0.548	0.550	2.009	−0.389	0.443	0.949	−0.386	−0.210

Source: Authors’ own work