I couldn't find anything close to this issue in order to solve it.
I'm using the Animation with moving objects II procedure to render an animation. Everything was working fine until I started the last step of the process that is loading the individual Irr Maps created in the Prepass. There are 96 frames. Suddenly, one ot the three objects I have in the scene, started to render wrong, no artifacts or splotches, but the object begun to look almost as if it were self-iluminated, no lighting decay towards the floor. The thing is that if I stop and begin the render at that frame, it renders correctly, but in the next frame the issue appears again. All the frames render correctly individually, but if I render a sequence, there you have it, the crazy stuff appears again.
I made an experiment just turning of GI, and the issue was gone. I also enabled Light Cache for every frame and the issue remained. So, those two things mean that the problem is with Irradiance Map.