Learning Forum

[diett004]

We are running high-RAM simulations (~1.5 TB) at the Minnesota Supercomputing Institute. The simulations tend to crash repeatedly, usually citing insufficient RAM. However, the same simulations do manage to complete intermittently. When they complete, HFSS consistently records a RAM value hundreds of GB below our actual allocation limit (2 TB). Our question: why would the simulations crash so often due to insufficient RAM when they don't end up requiring more than is available?

The only trend we notice is that higher frequency simulations (which usually require a finer mesh) tend to crash more frequently than lower frequency runs. However, as stated, when higher frequency simulations do successfully complete, they do not require much more RAM than lower frequency ones.

 

[DMARATHE]

Hi,

Thanks for putting up your query on learning forum. It very difficult to comment without looking at profile of simulation run and model complexity and settings.

However, I can suggest you some settings, that may be of some help

In Analysis setup, go for Mixed Order basis function and iterative solver which sometimes uses less RAM that Direct solver.

Thanks,

[diett004]

I tried switching to the iterative solver, and that does save quite a bit of RAM (as much as 400 GB), however the same problem still happens with somewhat finner meshes. It moves the bar up somewhat, but not enough. I have included the profile data from two different types of crashes. The first from before we switched to the iterative solver, and the second from after. Immediately below is the example of a crash using the direct solver. Note that it says "Out of RAM", but our actual ram limit was 1.97 TB, which is much higher than the 1.58 TB it lists using at the time of crashing. It is also important to note that simulations like this intermittently succeed with significantly less RAM than our limit (1.3-1.5 TB).

The next image shows a crash that happened after switching to the iterative solver. The profile shows almost no useful information:

[DMARATHE]

Hi,

I feel that your model is complex and requires very large computing resources. Based on first profile screenshot, Direct solver is asking for huge memory 3300 GB.

The second profile is suggesting that iterative solver is not able to converge.

Your problem requires deeper investigation by looking at model and complexity. If possible some complexities which will have less impact on EM simulations can be removed. However, this can be discussed if you have access to Ansys Technical Support and Iin that case support engineer can connect with you one on one to see the model and advice.

Thanks,