If moving the plates around tired me out to the point of impacting the session and/or my recovery I'd conclude that my work capacity needs improving, not that I need to minimise extraneous effort in the gym.