Dan Dvorscak
Ansys Employee

Yes, it is doing state-space fitting when it runs out of memory. Can you check to see if passivity enforcement is set in the s-parameter component in the Circuit schematic? To do this, right moust click on the component, select "Edit Model", and then check the passivity option in the Options tab. 
If that option is set to convex optimization instead of IFPV, that could explain the large memory usage here. Note this is separate from passivity enfrocement being applied during the SYZ sweep in SIwave.