The isotropic magneto-sensitive elastomeric composite (MEC) has been created by filling magnetically sensitive micro-sized carbonyl iron particles into a silicone rubber matrix. Compressive stress relaxation behavior of the isotropic MEC was investigated using the single relaxation test. Effects of different loading rates, constant strains, and external magnetic fields on the compressive stress relaxation of the isotropic MEC were studied. Results showed that the compressive stress relaxation of the isotropic MEC slightly depended on the loading rate, but was strongly dependent on the constant strain and the magnetic field. The compressive stress and relaxation modulus of the isotropic MEC increased with increasing the constant strain and magnetic field intensity as well. Besides, the stress relaxation response of the isotropic MEC in compression mode was examined using the four-parameter fractional derivative Zener model with the Mittag-Leffler function kernel. The model parameters were acquired by fitting the relaxation modulus to the experimental data of the isotropic MEC. The relaxation modulus and compressive stress with long-term predictions estimated from the investigated model were in very good agreement with the experimental data for the isotropic MEC at various loading rates, constant strains, and under different magnetic fields. In general, the studied model can be used to predict the long-term compressive stress relaxation of the isotropic MEC.