The issue with your definition can be shown when thinking about the absolute zero point (that is, 0°K): as per definition in this state, particles lack any kind of energy. Common tongue would say that it is "frozen". Now if this state lasts, let's say, indefinitely, time would be non-existent. As there is absolutely no change in the state of the particle.
PS: Although one could now argue that the act of observation changes its state.
As the universe approaches heat death, quantum mechanics dictates that particles jump between states randomly, such that passage of time cannot be measured meaningfully.
PS: Although one could now argue that the act of observation changes its state.