RSS feeds behavior analysis, structure and vocabulary
International Journal of Web Information Systems
ISSN: 1744-0084
Article publication date: 12 August 2014
Abstract
Purpose
The purpose of this paper is to present a thorough analysis of three complementary features of real-scale really simple syndication (RSS)/Atom feeds, namely, publication activity, items characteristics and their textual vocabulary, that the authors believe are crucial for emerging Web 2.0 applications. Previous works on RSS/Atom statistical characteristics do not provide a precise and updated characterization of feeds’ behavior and content, characterization that can be used to successfully benchmark the effectiveness and efficiency of various Web syndication processing/analysis techniques.
Design/methodology/approach
The authors empirical study relies on a large-scale testbed acquired over an eight-month campaign from 2010. They collected a total number of 10,794,285 items originating from 8,155 productive feeds. The authors deeply analyze feeds productivity (types and bandwidth), content (XML, text and duplicates) and textual content (vocabulary and buzz-words).
Findings
The findings of the study are as follows: 17 per cent of feeds produce 97 per cent of the items; a formal characterization of feeds publication rate conducted by using a modified power law; most popular textual elements are the title and description, with the average size of 52 terms; cumulative item size follows a lognormal distribution, varying greatly with feeds type; 47 per cent of the feed-published items share the same description; the vocabulary does not belong to Wordnet terms (4 per cent); characterization of vocabulary growth using Heaps’ laws and the number of occurrences by a stretched exponential distribution conducted; and ranking of terms does not significantly vary for frequent terms.
Research limitations/implications
Modeling dedicated Web applications capacities, Defining benchmarks, optimizing Publish/Subscribe index structures.
Practical implications
It especially opens many possibilities for tuning Web applications, like an RSS crawler designed with a resource allocator and a refreshing strategy based on the Gini values and evolution to predict bursts for each feed, according to their category and class for targeted feeds; an indexing structure which matches textual items’ content, which takes into account item size according to targeted feeds, size of the vocabulary and term occurrences, updates of the vocabulary and evolution of term ranks, typos and misspelling correction; filtering by pruning items for content duplicates of different feeds and correlation of terms to easily detect replicates.
Originality/value
A content-oriented analysis of dynamic Web information.
Keywords
Acknowledgements
Michel Scholl passed away on November 15, 2011, too early after a short battle with cancer. The authors would like to express their gratitude for everything he offered us on personal and professional levels during our long-lasting collaboration.
This work is supported by the French ANR project RoSeS (ANR-07-MDCO-011). This work is an enhanced version with a deeper analysis of the publication, “Characterizing Web Syndication Behavior and Content”, by Z. Hmedeh, N. Vouzoukidou, N. Travers, V. Christophides, C. du Mouza and M. Scholl, in Lecture Notes in Computer Science, Volume 6,997, 2011, pp 29-42 from The 12th International Conference on Web Information System Engineering (WISE’11).
Citation
Travers, N., Hmedeh, Z., Vouzoukidou, N., du Mouza, C., Christophides, V. and Scholl, M. (2014), "RSS feeds behavior analysis, structure and vocabulary", International Journal of Web Information Systems, Vol. 10 No. 3, pp. 291-320. https://doi.org/10.1108/IJWIS-06-2014-0023
Publisher
:Emerald Group Publishing Limited
Copyright © 2014, Emerald Group Publishing Limited