Text this: On higher-spin points and infinite distances in conformal manifolds