A comprehensive framework for modeling elasto-viscoplasticity in complex fluids is discussed. It is based on the plasticity mechanism of kinematic hardening, which is widely accepted in solid mechanics and accounts for transient yielding processes. We discuss a simple one dimensional variant of the model, as well as a fully three-dimensional, frame-invariant and thermodynamically admissible version of the model. Predictions for several canonical rheometric test protocols are provided. We also discuss possible extensions to account for additional rheological complexities exhibited by real fluids, such as thixotropy, nonlinear elasticity and normal stress differences. We find that this framework has several advantages over the more commonly used elastic Bingham-like or elastic Herschel Bulkley models for describing elasto-viscoplasticity. First, the model can account for behavior over a much wider range of viscometric test conditions. Second, it eliminates the flow/no flow criterion inherent in Bingham-like constitutive laws, which frequently requires regularization. Third, it is a flexible framework and allows for implementation of additional complexities, including thixotropic behavior and other nonlinear rheological features.