Small steps in the right direction are better than big steps in the wrong direction