I don’t know much about SOCs I2S, we have not yet designed any source so we have never investigated the possible ways.
Anyway, when the FIFO is on the input is a very high load, several Mohms.
If I have undesrtood your problem, I think you have to power off the FIFO and then the SOCs I2S.
The relays of the FIFO have nothing to do with the inputs, the switch between the two sample rate families (oscillators connected to the FIFO).
You can report your experience here and on diyaudio.com, no problem.