Ok I found the bug.
I noticed that the warm light is bright for around 0.5 seconds then it is switching to a much lower brightness (which is probably why the picture goes black as the exposure is set to the former brightness).
It is happening when I am changing the IVS rule, for instance for a different planning.
However if after changing the IVS rule, I visit (visit only, not changing anything) the Camera/Image/Illuminator settings (in my case for the night settings) and wait for the preview picture to appear, then the bug goes away.
Not sure if it goes away permanently, I'll have to check.
But at least I can confirm that this is a bug.
EDIT: unfortunately the "fix" did not survive the night. In fact it doesn't survive after 5-10 minutes. I reset all settings to default to see if the bug goes away, we'll see tonight.