To take the back track

to retrace one's steps; to retreat.

See also: Back

Webster's Revised Unabridged Dictionary, published 1913 by G. & C. Merriam Co.