Text this: Non-Markovian dynamics under time-translation symmetry