SMS versenden: Engineering a noise-free quantum memory for temporal mode manipulation