To read this content please select one of the options below:

RSS feeds behavior analysis, structure and vocabulary

Nicolas Travers (CEDRIC Laboratory, CNAM, Paris, France)
Zeinab Hmedeh (CEDRIC Laboratory, CNAM, Paris, France)
Nelly Vouzoukidou (LiP6 Laboratory, Pierre et Marie Curie University, Paris, France)
Cedric du Mouza (CEDRIC Laboratory, CNAM, Paris, France)
Vassilis Christophides (ICS-FORTH, University of Crete, Heraklion, Greece)
Michel Scholl (CEDRIC Laboratory, CNAM, Paris, France)

International Journal of Web Information Systems

ISSN: 1744-0084

Article publication date: 12 August 2014




The purpose of this paper is to present a thorough analysis of three complementary features of real-scale really simple syndication (RSS)/Atom feeds, namely, publication activity, items characteristics and their textual vocabulary, that the authors believe are crucial for emerging Web 2.0 applications. Previous works on RSS/Atom statistical characteristics do not provide a precise and updated characterization of feeds’ behavior and content, characterization that can be used to successfully benchmark the effectiveness and efficiency of various Web syndication processing/analysis techniques.


The authors empirical study relies on a large-scale testbed acquired over an eight-month campaign from 2010. They collected a total number of 10,794,285 items originating from 8,155 productive feeds. The authors deeply analyze feeds productivity (types and bandwidth), content (XML, text and duplicates) and textual content (vocabulary and buzz-words).


The findings of the study are as follows: 17 per cent of feeds produce 97 per cent of the items; a formal characterization of feeds publication rate conducted by using a modified power law; most popular textual elements are the title and description, with the average size of 52 terms; cumulative item size follows a lognormal distribution, varying greatly with feeds type; 47 per cent of the feed-published items share the same description; the vocabulary does not belong to Wordnet terms (4 per cent); characterization of vocabulary growth using Heaps’ laws and the number of occurrences by a stretched exponential distribution conducted; and ranking of terms does not significantly vary for frequent terms.

Research limitations/implications

Modeling dedicated Web applications capacities, Defining benchmarks, optimizing Publish/Subscribe index structures.

Practical implications

It especially opens many possibilities for tuning Web applications, like an RSS crawler designed with a resource allocator and a refreshing strategy based on the Gini values and evolution to predict bursts for each feed, according to their category and class for targeted feeds; an indexing structure which matches textual items’ content, which takes into account item size according to targeted feeds, size of the vocabulary and term occurrences, updates of the vocabulary and evolution of term ranks, typos and misspelling correction; filtering by pruning items for content duplicates of different feeds and correlation of terms to easily detect replicates.


A content-oriented analysis of dynamic Web information.



Michel Scholl passed away on November 15, 2011, too early after a short battle with cancer. The authors would like to express their gratitude for everything he offered us on personal and professional levels during our long-lasting collaboration.

This work is supported by the French ANR project RoSeS (ANR-07-MDCO-011). This work is an enhanced version with a deeper analysis of the publication, “Characterizing Web Syndication Behavior and Content”, by Z. Hmedeh, N. Vouzoukidou, N. Travers, V. Christophides, C. du Mouza and M. Scholl, in Lecture Notes in Computer Science, Volume 6,997, 2011, pp 29-42 from The 12th International Conference on Web Information System Engineering (WISE’11).


Travers, N., Hmedeh, Z., Vouzoukidou, N., du Mouza, C., Christophides, V. and Scholl, M. (2014), "RSS feeds behavior analysis, structure and vocabulary", International Journal of Web Information Systems, Vol. 10 No. 3, pp. 291-320.



Emerald Group Publishing Limited

Copyright © 2014, Emerald Group Publishing Limited

Related articles