The discrete Fourier transformation for seasonality and anomaly detection of an application to rare data

Aryana Collins Jackson (Department of Computer Science, École Nationale d’Ingénieurs de Brest, Brest, France) (Department of Mathematics, Cork Institute of Technology, Cork, Ireland)
Seán Lacey (Department of Mathematics, Cork Institute of Technology, Cork, Ireland)

Data Technologies and Applications

ISSN: 2514-9288

Publication date: 21 May 2020

Abstract

Purpose

The discrete Fourier transformation (DFT) has been proven to be a successful method for determining whether a discrete time series is seasonal and, if so, for detecting the period. This paper deals exclusively with rare data, in which instances occur periodically at a low frequency.

Design/methodology/approach

Data based on real-world situations is simulated for analysis.

Findings

Cycle number detection is done with spectral analysis, period detection is completed using DFT coefficients and signal shifts in the time domain are found using the convolution theorem. Additionally, a new method for detecting anomalies in binary, rare data is presented: the sum of distances. Using this method, expected events which have not occurred and unexpected events which have occurred at various sampling frequencies can be detected. Anomalies which are not considered outliers to be found.

Research limitations/implications

Aliasing can contribute to extra frequencies which point to extra periods in the time domain. This can be reduced or removed with techniques such as windowing. In future work, this will be explored.

Practical implications

Applications include determining seasonality and thus investigating the underlying causes of hard drive failure, power outages and other undesired events. This work will also lend itself well to finding patterns among missing desired events, such as a scheduled hard drive backup or an employee's regular login to a server.

Originality/value

This paper has shown how seasonality and anomalies are successfully detected in seasonal, discrete, rare and binary data. Previously, the DFT has only been used for non-rare data.

Keywords

Acknowledgements

Funding: This research did not receive any specific grant from funding agencies in the public, commercial or not-for-profit sectors.This work was supported by Department of Mathematics, Cork Institute of Technology.

Citation

Collins Jackson, A. and Lacey, S. (2020), "The discrete Fourier transformation for seasonality and anomaly detection of an application to rare data", Data Technologies and Applications, Vol. 54 No. 2, pp. 121-132. https://doi.org/10.1108/DTA-12-2019-0243

Download as .RIS

Publisher

:

Emerald Publishing Limited

Copyright © 2020, Emerald Publishing Limited

To read the full version of this content please select one of the options below

You may be able to access this content by logging in via Shibboleth, Open Athens or with your Emerald account.
To rent this content from Deepdyve, please click the button.
If you think you should have access to this content, click the button to contact our support team.