This paper presents a novel approach to Lyapunov stability theory-based adaptive filter (LAF) design. The proposed design is based on the minimization of the Euclidean norm of the difference weight vector under negative definiteness constraint defined over a novel linear Lyapunov function. The proposed fixed step size LAF (FSS-LAF) algorithm is first obtained by using the method of Lagrangian multipliers. The FSS-LAF satisfying asymptotic stability in the sense of Lyapunov provides a significant performance gain in the presence of a measurement noise. The stability of the FSS-LAF algorithm is also statistically analyzed in this study. Moreover, gradient variable step size (VSS) algorithms are adapted to the FSS-LAF algorithm to further enhance the performance for the first time in this paper. These VSS algorithms are Benveniste (BVSS), Mathews and Farhang-Ang (FVSS) algorithms. Simulation results on system identification problems show that the bounds of step size for the FSS-LAF algorithm are verified, and especially, the BVSS-LAF and FVSS-LAF algorithms provide a better trade-off between steady-state mean square deviation error and convergence rate than other proposed algorithms.