TY - JOUR AB - Purpose Adaptive equalization plays an important role in digital communication to reduce the distortions due to inter-symbol interference. An adaptive filter is used as an equalizer model in channel equalization. An adaptive algorithm is the heart of the adaptive filter which finds the optimum coefficients of the filter. The choice of the adaptive algorithm improves the convergence rate and minimizes the mean square error (MSE). This paper aims to propose a cat swarm optimization (CSO)-based adaptive algorithm and its modification to improve the performance of a channel equalizer.Design/methodology/approach The input digital training data are transmitted through different channel conditions. A linear transversal filter is used as a channel and equalizer model. The equalizer coefficients are trained by the proposed simplified cat swarm optimization (SCSO) algorithm to find the estimated digital training data.Findings The performance of the proposed SCSO algorithm is compared with particle swarm optimization (PSO)-based channel equalization. The improvement in convergence rate and MSE is verified under linear and nonlinear channel conditions with different delay spreads. The optimum parameters of the SCSO are found using simulation-based sensitivity analysis.Originality/value This paper analyzes a CSO algorithm for adaptive channel equalization and proposes a SCSO algorithm to identify the optimum coefficients of a transversal equalizer. The seeking mode process is simplified in the proposed SCSO to achieve better performance in channel equalization. The proposed SCSO algorithm guarantees minimum MSE in all independent runs, whereas in PSO, few misses are possible. VL - 36 IS - 1 SN - 0332-1649 DO - 10.1108/COMPEL-08-2015-0289 UR - https://doi.org/10.1108/COMPEL-08-2015-0289 AU - D.C. Diana AU - S.P. Joy Vasantha Rani PY - 2017 Y1 - 2017/01/01 TI - Novel cat swarm optimization algorithm to enhance channel equalization T2 - COMPEL - The international journal for computation and mathematics in electrical and electronic engineering PB - Emerald Publishing Limited SP - 350 EP - 363 Y2 - 2024/04/23 ER -