Text this: On affine groups admitting invariant two-point sets