I stumbled on a workaround.
When it worked like @sukitha.jayasinghe’s screenshot, I was using
// data is YUVRawDataI420;
stream.Write(data.GetBuffer());
If I write this instead, it works as expected.
stream.Write(data.GetYBuffer());
stream.Write(data.GetUBuffer());
stream.Write(data.GetVBuffer());