# Making sense of the changing face of Google’s search engine results page: an advertiser’s perspective

Divya Sharma (Indian Institute of Management Rohtak, Rohtak, Haryana, India)
Agam Gupta (Indian Institute of Management Rohtak, Rohtak, Haryana, India)
Arqum Mateen (Indian Institute of Management Kozhikode, Kozhikode, India)
Sankalp Pratap (Indian Institute of Management Tiruchirappalli, Tiruchirappalli, Tamil Nadu, India)

ISSN: 1477-996X

Publication date: 12 March 2018

## Abstract

### Purpose

Google commands approximately 70 per cent of search market share worldwide, resulting in businesses investing heavily in search engine advertising on Google to target potential customers. Recently, Google changed the way in which content and ads were displayed on the search engine results page. This reshuffling of content and ads is expected to affect the advertisers who advertise on Google and/or use it to drive traffic to their websites. The purpose of this study is to analyze the impact of these changes on various stakeholders.

### Design/methodology/approach

Data have been collected from various sources on the internet including blogs and discussion forums. Netnography has been used as it allows a detailed evaluation of the consumers’ needs, wants and choices in a virtual space.

### Findings

The average cost-per-click for ads on the top positions is expected to increase. Advertisers whose ads usually occupy the lower positions would be adversely affected. To counter this, more emphasis should be placed on ad extensions and on product listing ads. In addition, organizations would benefit from increased efforts on search engine optimization.

### Practical implications

A variety of coping strategies have been developed that can help marketers to successfully navigate through the change, including the use of ad extensions and the use of product listing ads.

### Originality/value

This practice-focused paper offers guidelines for digital marketers to use sponsored search more effectively as part of their arsenal in light of some important changes recently made by Google. The potential of netnography as a research methodology has also been expanded by using it in a novel setting and in drawing up actionable insights.

## Keywords

#### Citation

Sharma, D., Gupta, A., Mateen, A. and Pratap, S. (2018), "Making sense of the changing face of Google’s search engine results page: an advertiser’s perspective", Journal of Information, Communication and Ethics in Society, Vol. 16 No. 1, pp. 90-107. https://doi.org/10.1108/JICES-06-2017-0035

### Publisher

:

Emerald Publishing Limited

## 1. Introduction

Increased adoption of mobile and communication technologies along with the proliferation of online marketplaces has significantly altered the competitive landscape of many industries. Companies have discovered new ways of reaching out to customers leading to new opportunities in marketing (Chong et al., 2010). Prominent factors affecting such developments and giving rise to the primacy of online mediums for advertising include wide geographical reach (Laudon and Laudon, 2002), usage convenience and interactive communication ability (Chaffey, 2004). One of the ways in which the internet has enabled marketers is through search engine advertising (Edelman et al., 2007). In this study, we have investigated how changes to the way information is organized and presented on the search engine results page (SERP) may affect an advertiser. An SERP is the page that displays ads and other informational content in the form of links when a user makes a search query to the search engine.

In early 2016, Google announced a change in the layout of its SERP. This means that the way in which sponsored search ads appear when a user queries Google has changed. When a user queries a search engine on the SERP, sponsored search ads are displayed as part of paid listing alongside organic (unpaid) listing. The change has resulted in many digital advertising agencies, managers and marketers talking, debating and conjecturing about the possible impacts of the change.

Given the salience of SSA not only to the large advertisers but also to the small- and medium-scale advertisers, it becomes important to understand how the change made by Google in its ad layout on the SERP, with respect to both organic and paid listings, is likely to affect the advertisers. Specifically, we have attempted to answer the following research questions in light of the changes made to the Google SERP:

RQ1.

What would be the impact on the costs and bidding strategies of advertisers using the medium?

RQ2.

How would the organic listings on the SERP get affected?

RQ3.

How can advertisers respond to these changes?

Adopting a netnographic research method, we have conducted a thorough analysis of blog posts, comments on blog posts, Facebook posts and related comments and opinion pieces, and independent analysis of real data, published on the internet with respect to this change. Based on this analysis, we have presented the possible impacts of the change on advertisers, identified new possibilities for advertisers in light of the changes and suggested an agenda for future research. However, before presenting our analysis, we will delineate the changes made by Google.

## 2. Changes in the search engine results page

The change in Google’s ad layout has altered the position, placement and the number of ads that would appear on the SERP when browsing on desktops. Figure 1 illustrates an SERP along with the ads and the organic listings prior to the change. The top three ads were displayed on the top of the SERP, followed by the next eight ads on the right-side panel.

## 3. Literature review

A change in the Google ad layout on the search engine provides a natural experiment to understand the effects of the placement of advertisement on the advertisers’ metrics, the importance of organic listings and on the search engine revenue. The change as explained before may be summarized as a change in the number of ads on the page and a change in the layout that moves some ads from the right side to the left and some from the top of the page to the bottom. Thus, we specifically focus on the sponsored search literature that has dealt with ad positions and interplay of organic and paid listings.

One of the key variables that impacts important search engine advertising metrics such as click-through rates (CTRs), conversion rates and CPC is the ranking of advertisements (Ghose and Yang, 2009; Yoo, 2012). An early research found that the ordinal ranking of firms in an ordered listing on Amazon’s online marketplace is strongly related to a consumer’s likelihood of clicking on the links to the firms’ offerings in the listing and purchasing from it (Smith and Brynjolfsson, 2001). Search engines, more specifically SERP, also create an environment where the order in which content is presented limits the freedom and flexibility of a consumer while browsing organic and paid content. An SERP is an ordered or directional market, wherein the order in which users browse content is a function of the rank of the content on the page (Arbatskaya, 2007). Such directionality is an outcome of cognitive burden as it is cognitively “costlier” for a consumer to visit the links (organic/ads) at the bottom of a list in comparison to visiting those at the top of the list (Animesh et al., 2011; Sherman, 2005). As a result, on an SERP, the content that is placed at the top attracts greater user attention compared to content that is placed at the bottom of the page. It is not surprising, then, that any change in the layout, as implemented by Google on its SERP, would affect the ability of the listed firms’ to attract search engine users. Hence, it becomes important to understand the impact of this change in ordering and positioning on the businesses. Previous studies such as those by Agarwal et al. (2011) and Animesh et al. (2011) use consumer search theory and positioning strategies to empirically analyze the impact of competition, ad rank and the unique selling proposition on the CTR.

Research has also suggested that there is a positive interdependence between organic listings and paid search, and textual similarity between organic and paid listings is likely to affect the perceived usefulness of the ads (Danescu-Niculescu-Mizil, 2010). The Pew Internet and American Life Project (Fallows, 2005) reported that 38 per cent of searchers reported that they were unaware of the distinction between sponsored links and non-sponsored links; however, research also suggests that there is a negative bias toward the ads on the search engine (Jansen and Spink, 2007).

The review of literature indicates that the while extant literature provides us with important constructs that can possibly help us understanding the impact of changes in SERP layout, no research has specifically looked at this problem. Given the importance of sponsored search to advertisers and Google in terms of revenue, in this paper, we build on the extant literature to understand how some of the key metrics on search engines are likely to be affected through a conceptual model and research propositions.

## 4. Method

The purpose of this research is to analyze the possible effects of Google’s change in the layout of its SERP on advertisers. To this end, we wanted to understand the reactions of advertisers to this change. The internet as a repository of information and varied opinions, often available publically on social networking websites, blogs and forums, provides us with data resources that could be used to understand advertisers’ concerns, analysis and views regarding this change. The use of content from the Web for conducting research has been emphasized for conducting marketing research in the information economy characterizing the current era (Hansson et al., 2013). We use netnography (Kozinets, 2002), which is defined as a written account resulting from fieldwork studying the communities that emerge from online, computer-mediated or internet-based communications. Netnography is an unobtrusive, naturalistic method of inquiry permitting the investigation of focal online communities without researchers’ participation or interference in the community members’ activities (Kozinets, 2002; Cova and Pace, 2006). As a technique, it allows gathering qualitative insights into consumers’ needs, wants and choices in a virtual space, and symbolic meanings in the case of online internet networks and communities. At the same time, it is faster, simpler, timelier and less expensive than traditional ethnography (Kozinets, 2006). Stemming from these advantages, it allows the researcher to understand the hidden dimensions of consumers (Rageh et al., 2013). As a result, it is argued that it can be an effective market research (Xun and Reynolds, 2010) and consumer education research tool (Sandlin, 2007). Netnography has been used to study tourist cultures (Wu and Pearce, 2014; Rageh et al., 2013; Nelson and Otnes, 2005), brand communities (Cova and Pace, 2006; Xun and Reynolds, 2010; Hollebeek et al., 2017) and value creation in banking services (Medberg and Heinonen, 2014), among others.

For the purpose of this research, we resort to non-participant netnography (Cova and Pace, 2006), wherein the researchers are not active participants in the online communities they use as resources but engage in sense making from those resources. To understand the possible impact of the change implemented in the ad layout of Google’s SERP, we begin by systematically assembling a repository of content related to facts about the changes, and views of various experts and marketers on the changes.

We focused on the three most popular sponsored search marketing blogs among search engine marketing practitioners around the world, namely, “searchengineland.com”, “searchenginewatch.com” and “searchenginejournal.com”. Within the selected forums, not only do the editors of the blogs but also the employees and the former employees of search engines and various top management individuals of firms connected within the search ecosystem contribute articles and content. The blogs also post their articles on social media websites such as Facebook, where a number of individual marketing managers and small business owners read the articles and engage in discussion. The opinions of various stakeholders about the change were, therefore, available to us for analysis. In addition to these blogs, some other websites that published initial analysis of the change, based on the primary data of the accounts being managed, were also chosen as sites for data collection. For example, the chief technology officer (CTO) of a popular digital advertising agency looked at the data of its clients and presented an analysis using thousands of accounts across all industries.

We also visited popular Q&A websites such as Quora.com and analyzed questions with reference to this change. In addition, we performed several Google searches to ensure that we had covered as many comments and debates as possible on this topic. Following ethical research practices, we have anonymized the details about persons whose comments/anecdotes on online forums are quoted in this work. However, we would like to underscore that all data used for the purpose of this research are publicly available and do not require either sign-up to a particular website or seeking permission before use. A summary of the websites and articles used for this research is present in Table I.

After the collection of data, the authors resorted to a manual approach to analyze the data. Manual approach is recommended when the data set is reasonable in size, can be organized quite readily and the researchers are very familiar with the context (Kozinets, 2010). The prior industry experience of one of the researchers in managing search engine marketing activities, along with prior research in this area by the authors, proved to be helpful in this regard.

## 5. Emerging themes and research propositions

An SERP consists of two distinct components: organic search listings and paid search listings. Often marketers invest considerably in both these components. To manage the former, advertisers engage in search engine optimization, whereby they spend considerable time and effort in making their websites’ search engine friendly such that the websites rank high on the SERP’s organic listings. For the latter, advertisers may spend money to participate in SSA. Appearing on the same page, organic listings and paid listings are so interconnected that an analysis of the change would be incomplete without considering its effect on organic listings. Therefore, we analyze the impact of the change separately on each of these. In addition, the change is also likely to alter the search engine user behavior. We present our findings beginning with the impact on paid search listings, followed by the impact on organic listings and ending with a discussion on the potential alterations in user behavior as a result of the changes in the SERP layout.

### 5.1 Effect on paid search

#### 5.1.1 Qualitatively better ad inventory available but fewer number of advertisers on search engine results page.

An analysis by an ex-employee of Google in a blog suggests that at an aggregate level, net of the pluses and the minuses, “[t]he change makes 18 per cent more clicks available”. However, it is important to note that this analysis is made on an aggregate level and individual advertisers are likely to have different ad positions for different keywords. While the advertiser will benefit by using some of these keywords, they will be adversely affected by others. The exact effect for an advertiser would, therefore, also depend on the portfolio of keywords the advertiser possesses and the average positions of ads for those keywords:

P1.

The shift in positioning of ads from the right panel to the left will significantly improve the performance of ads in terms of CTR, especially for those advertisers who were previously ranked fourth on the SERP.

P2.

The reduction in number of ads on the SERP from a maximum of eleven to a maximum of seven will adversely affect advertisers whose ads occupied Positions 8-11 previously.

#### 5.1.2 Better information quality.

P3.

The reduction in number of ads on the SERP from a maximum of eleven to a maximum of seven will improve the accuracy of average ad position reported by Google to advertisers, through a reduction in the deviation of possible ad positions.

#### 5.1.3 A shift in most profitable positions.

Empirical research has found that top positions are not the most profitable keywords for advertisers. While top positions ascertain a higher CTR and a greater absolute number of clicks, it is the middle positions that have the best profits, because of lower costs in comparison to the top positions (Ghose and Yang, 2009). A reorganization of these ads, from the side panel to the bottom of the page, will lead to a decrease in the number of conversions because of the decreased impressions. A decrease in the number of low-cost acquisitions may increase the overall account-level acquisition costs for the advertisers, and could also adversely affect bids and positions of other keywords in the advertisers’ portfolios. This view is reflected in one of the search engine advertising agency’s (lunametric) blogs:

A positive side effect is increased overall audience reach. With more ads at the top of the page, you should expect to have a high level of interaction with your targeted audience[…]. The only potential negative effect is the one you have already mentioned – expected cost increases:

P4.

The shift in positioning of ads from the right panel to the left will lead to a decline in the profitability of ads that earlier occupied the right panel because of lower impressions at the bottom of the SERP.

#### 5.1.4 Possibility of cost-per-click rising.

There are two reasons for an expectation of average CPC rising in the wake of the change implemented. First, because the ads ranked 5 to 7 will now be moved to the bottom of the page, there might be a decrease in the impression share of these ads, as explicated earlier. This will force the advertisers with ads in these positions to target higher positions by bidding higher, consequently increasing the CPCs. In addition, the top three (four) positions will have disproportionate visibility compared to the bottom three. This may increase the inherent valuation of these slots for advertisers, specifically for branded search queries. Second, advertisers placed on Positions 8 through 11 will no longer appear on the primary SERP. It is expected that they will compete for positions on the first page by bidding higher. As a result, CPCs for advertisers on an average are expected to rise. The increase in CPCs was the most common belief that advertisers/agency personnel/individuals held with respect to this change. Some of these comments from social media users (SMUs) in a discussion are presented below:

SMU 1: Whoa whoa […] that means only top 3? Cpc are gonna shoot up!

SMU 2: I could see tens of thousands of small businesses going into financial trouble as a result of this decision.

SMU 3: As a small business owner I use Adwords and to hear that it has been cancelled leaves me with a challenge. How do I compete now with large companies whose budget is higher or[far] more to bid? The answer: I cant! My ads are usually top 3 on the right so the ROI is decent enough to get visitors to my site and generate, perhaps another solution to small business owners should have been looked at! What happens to the right side? How do my clients find me/us, do I stop using Adwords if I cant compete? Kick the small guys out and the rich get richer […] […].

Therefore, we propose the following:

P5.

The shift in positioning of ads from the right panel to the left and the reduction in number of ads on the SERP from a maximum of eleven to a maximum of seven will, on an average, lead to rise in CPC for advertisers.

Based on the above analysis, Figure 3 presents the impact of the change in ad layout on Google’s SERP on ad impressions, CTR and CPC for different ad positions.

### 5.2 Effect on organic listings

We now focus our attention on the organic listings on the SERP. Given that above the line the page size is constant, increasing the prominence of paid listings would inadvertently come at the cost of organic listing. With the number of ads above the organic listings increasing to 4, the organic listings would be pushed down the page and only one organic listing would be seen above the fold (without having to scroll down). This compromises the salience of the real estate available for organic listings. Our analysis reveals that it is almost unanimously believed that the biggest looser of this change is organic listing. Therefore, we propose that:

P6.

The shift in positioning of ads from the right panel to the left, and the possibility of having four ads appear above the organic listings, will result in paid listings usurping the real estate that was previously allocated to organic listings, leading to a possible decrease in the number of clicks of the second and third organic listings.

The change may also have second-order effects, where a simultaneous presence of an advertiser in both organic and paid listing may have a reinforcing effect. This is because the CTR on organic listings has been shown to have a positive interdependence with CTR on paid listings, and vice versa (Ghose and Yang, 2009). Now that the ads and organic lists are all placed in one list, the observed reinforcement in CTR may be higher, thereby increasing the importance of top rank in organic listing.

In addition, while the SERP layout change appears to have a negative impact on organic listings in terms of their position on the SERP, the importance of organic listings is likely to increase for small- and medium-scale businesses, which may not be able to afford the high CPCs that result from the change. The increased importance of search engine optimization, that is, activities performed by marketers to rank high organically, in the wake of the changes is reflected in the comment posted by an SMU (probably a business owner) on a discussion page on Facebook:

SMU 4: So basically Google search results now look like Google’s mobile results. 4 sponsor links above the fold/scroll and 3 local maps results and a couple organic results. And old school SEO’s laughed at me when I started building and optimizing Google listings for businesses.

P7.

The possible increase in competition for top slots is likely to increase the CPCs for advertisers, thus intensifying the value and need for top position in organic listings especially for small- and medium-scale firms.

### 5.3 Potential changes in user behavior and its effects on advertisers

It would be naïve to imagine that the SERP layout change implemented by Google would impact the advertisers alone. The new format is also likely to affect users who scavenge for information on the internet by typing queries on search engines. The effect on users and their response to the change is likely to, in turn, impact the advertisers.

The change will make for a more homogenous viewing of the SERP across multiple devices, as the new layout means that there will be no difference from now onwards in the way the SERP renders on desktops and mobile devices. This is likely to reduce the cognitive strain that users otherwise face because of the layout and concomitant content differences across devices. Moreover, with an overall decrease in available ad slots, users can expect more relevant ads, that is, ads with higher QS, to get displayed. As a result, this could potentially lead to higher CTR. This is also reflected in a statement by an SMU on Facebook:

SMU 5: Now the entire game will be on the quality score [relevance] to match up the top spot […] […]. But the ask is will it also improve the CTR since we now only have 3 sweet spots?

## References

Abhishek, V., Hosanagar, K. and Fader, P.S. (2015), “Aggregation bias in sponsored search data: the curse and the cure”, Marketing Science, Vol. 34 No. 1, pp. 59-77.

Agarwal, A., Hosanagar, K. and Smith, M.D. (2011), “Location, location, location: an analysis of profitability of position in online advertising markets”, Journal of Marketing Research, Vol. 48 No. 6, pp. 1057-1073.

Allen, J. (2010), “How much is a Google top spot worth?”, Search Engine Watch,

Animesh, A., Viswanathan, S. and Agarwal, R. (2011), “Competing ‘creatively’ in sponsored search markets: the effect of rank, differentiation strategy, and competition on performance”, Information Systems Research, Vol. 22 No. 1, pp. 153-169, doi: 10.1287/isre.1090.0254.

Arbatskaya, M. (2007), “Ordered search”, The RAND Journal of Economics, Vol. 38 No. 1, pp. 119-126,

Carlson, M. (2014), “When news sites go native: redefining the advertising – editorial divide in response to native advertising”, Journalism, Vol. 7 No. 16, pp. 849-865, doi: 10.1177/1464884914545441.

Chaffey, D. (2004), E-Business and E-Commerce Management, Financial Times-Prentice Hall, London, UK.

Chan, T.Y., Wu, C. and Xie, Y. (2011), “Measuring the lifetime value of customers acquired from Google search advertising”, Marketing Science, Vol. 30 No. 5, pp. 837-850,

Cova, B. and Pace, S. (2006), “Brand community of convenience products: new forms of customer empowerment – the case ‘my nutella the community”, European Journal of Marketing, Vol. 40 Nos 9/10, pp. 1087-1105, doi: 10.1108/03090560610681023.

Danescu-Niculescu-Mizil, C. (2010), “Competing for users’ attention: on the interplay between organic and sponsored search results”, Proceedings of the 19th International Conference on World Wide Web - WWW ‘10, pp. 291-300, available at: http://dl.acm.org/citation.cfm?id=1772721

Economist (2014), “Should digital monopolies be broken up?”, November,

Edelman, B., Ostrovsky, M. and Schwarz, M. (2007), “Internet advertising and the generalized second-price auction: selling billions of dollars”, The American Economic Review, Vol. 97 No. 1, pp. 242-259.

Fallows, D. (2005), Search Engine Users, Pew Internet and American Life Project Report,

Ghose, A. and Yang, S. (2009), “An empirical analysis of search engine advertising: sponsored search in electronic markets”, Management Science, Vol. 55 No. 10, pp. 1605-1622, doi: 10.1287/mnsc.1090.1054.

Google (2009), “Eye-tracking studies: more than meets the eye”, Google Official Blog.

Gupta, A. and Mateen, A. (2014), “Exploring the factors affecting sponsored search ad performance”, Marketing Intelligence & Planning, Vol. 32 No. 5, pp. 586-599, doi: 10.1108/MIP-05-2013-0083.

Hansson, L., Wrangmo, A. and Søilen, K.S. (2013), “Optimal ways for companies to use Facebook as a marketing channel”, Journal of Information, Communication and Ethics in Society, Vol. 11 No. 2, pp. 112-126.

Hollebeek, L.D., Juric, B. and Tang, W. (2017), “Virtual Brand community engagement practices: a refined typology and model”, Journal of Services Marketing, Vol. 31 No. 3, pp. 204-217, doi: 10.1108/JSM-01-2016-0006.

Jansen, B.J. and Spink, A. (2007), “The effect on click-through of combining sponsored and non-sponsored search engine results in a single listing”, Proceedings of the 2007 Workshop on Sponsored Search Auctions, WWW Conference, available at: www.wwwconference.org/www2007/workshops/paper_96.pdf

Kian Chong, W., Shafaghi, M., Woollaston, C. and Lui, V. (2010), “B2B E‐marketplace: an E‐marketing framework for B2B commerce”, Marketing Intelligence & Planning, Vol. 28 No. 3, pp. 310-329, doi: 10.1108/02634501011041444.

Kim, L. (2016), “Google’s right-side adpocalypse: what really happened (DATA)”, available at: http://searchengineland.com

Kozinets, R.V. (2006), “Click to connect: netnography and tribal advertising”, Journal of Advertising Research, Vol. 46 No. 3, pp. 279-288, doi: 10.2501/S0021849906060338.

Kozinets, R.V. (2002), “The field behind the screen: using netnography for marketing research in online communities”, Journal of Marketing Research, Vol. 39 No. 1, pp. 61-72.

Kozinets, R.V. (2010), Netnography: Doing Ethnographic Research Online, Sage Publications, California.

Laudon, K.C. and Laudon, J.P. (2002), Management Information Systems: Managing the Digital Firm, Prentice Hall, New Jersey.

Lee, J. (2014), “SEM the most effective acquisition channel for 85% of retailers”, Search Engine Watch.

Li, H. and Bukovac, J.L. (1999), “Characteristics”, Journalism & Mass Communication Quarterly, Vol. 76 No. 2, pp. 341-353.

MarketingCharts_Staff (2012), “Traditional media not a good source of leads, say B2B marketers”, MarketingCharts.com,

Medberg, G. and Heinonen, K. (2014), “Invisible value formation: a netnography in retail banking”, Dr Heinonen, K. (Ed.) International Journal of Bank Marketing, Vol. 32 No. 6, pp. 590-607, doi: 10.1108/IJBM-03-2014-0041.

Mohan, N. (2013), “The importance of being seen: viewability and brands”, Adwords.blogspot.in.

Nelson, M.R. and Otnes, C.C. (2005), “Exploring cross-cultural ambivalence: a netnography of intercultural wedding message boards”, Journal of Business Research, Vol. 58 No. 1 SPEC.ISS, pp. 89-95, doi: 10.1016/S0148-2963(02)00477-0.

PricewaterhouseCoopers (2016), “IAB internet advertising revenue report – 2015 full year results”, Interactive Advertising Bureau Report,

Rageh, A., Melewar, T.C. and Woodside, A. (2013), “Using netnography research method to reveal the underlying dimensions of the customer/tourist experience”, Qualitative Market Research: An International Journal, Vol. 16 No. 2, pp. 126-149, doi: 10.1108/13522751311317558.

Rosso, M. and Jansen, B.J. (2010), “Brand names as keywords in sponsored search advertising”, Communications of the Association for Information Systems, Vol. 27 No. 1, pp. 81-98.

Sandbu, M. (2017), “The systemic reach of big tech”, Financial Times, July 4,

Sandlin, J.A. (2007), “Netnography as a consumer education research tool”, International Journal of Consumer Studies, Vol. 31 No. 3, pp. 288-294, doi: 10.1111/j.1470-6431.2006.00550.x.

Sayedi, A., Jerath, K. and Srinivasan, K. (2014), “Competitive poaching in sponsored search advertising and its strategic impact on traditional advertising”, Marketing Science, Vol. 33 No. 4, pp. 586-608,

Sherman, C. (2005), “A new F-word for Google search results”, available at: www.webcitation.org/5FmwyPgDv

Smith, M.D. and Brynjolfsson, E. (2001), “Consumer decision-making at an internet shopbot: brand still matters”, The Journal of Industrial Economics, Vol. 49 No. 4, pp. 541-558.

Speck, P.S. and Elliott, M.T. (1997), “The antecedents and consequences of perceived advertising clutter”, Journal of Current Issues & Research in Advertising, Vol. 19 No. 2, pp. 39-54, doi: 10.1080/10641734.1997.10524436.

van Deursen, A.J.A.M. and van Dijk, J.A.G.M. (2009), “Using the internet: skill related problems in users’ online behavior”, Interacting with Computers, Elsevier B.V, Vol. 21 Nos 5/6, pp. 393-402, doi: 10.1016/j.intcom.2009.06.005, .

Wu, M.-Y. and Pearce, P.L. (2014), “Appraising netnography: towards insights about new markets in the digital tourist era”, Current Issues in Tourism, Vol. 17 No. 5, pp. 463-474, doi: 10.1080/13683500.2013.833179.

Xun, J. and Reynolds, J. (2010), “Applying netnography to market research: the case of the online forum”, Journal of Targeting, Measurement and Analysis for Marketing, Vol. 18 No. 1, pp. 17-31, doi: 10.1057/jt.2009.29.

Yoo, C.Y. (2012), “An experimental examination of factors affecting click-through of keyword search ads”, Journal of Current Issues & Research in Advertising, Vol. 33 No. 1, pp. 56-78, doi: 10.1080/10641734.2012.675559.

## Corresponding author

Arqum Mateen can be contacted at: arqumm@iimk.ac.in