CUDA Memory Error on 0.67 (Patreon)

Since there is a lot of reports on this one, decided to make a new post about it.

I'm doing some heavy debugging to find the source of the problem. There was some heavy changes to the .ckpt loader and Lora also do some changes to the weights, even if no Lora file is selected, so those are my best bets, but i'm still in the process of finding the problem. Thanks for all the feedback, it help me find the problem.

The main pipeline for image rendering is the same, so it should be a fixable problem.