Learning Forum

[lei2019]

Hi,

This is LES simulation of coal combustion, by species model, Discrete phase model, DO radiation; the particle number reaches up to 50 million;

using 2021R1, automatic Metis partitioning, single precision;

using two udf file to define coal properties wsg for radiation and define ash volume after coal burnout.

I use 120 processors to run this case by issuing command

/project/research/ansys/reserve_fluent.sh 120 

/project/research/ansys21/fluent21 3d -t120 -cnf=/home/research/username/fluenthosts.88888 -pib.ofed -gui_machine=chernobyl.engr.wustl.edu 

I issue these commands in VNC session using one processor. the system is

[Amine Ben Hadj Ali]

First of all running this kind of runs with single precision is a bad idea.
Second: you should keep track of memory usage of the case every couple of iterations using (proc-stats) and watch for excessive memory loading over time.
Third: Rough estimation for your case: you require a minimum of 75 GB RAM and even more for single precision: do all your 120 CPUS together offer that?

[lei2019]

the mesh is 3 million, and if using double precision, reading data file will encounter issues in ANSYS; H5 temporarily solve this issue. now the data file is about 12GB; if using DP, then it will double the size. and the particle parcel reaches about 50 million, which Fluent has difficulty handling.
The momery has been tracked by cluster administrator every tenth of a second; and no abnormal situation till now.

[lei2019]

[Amine Ben Hadj Ali]

So CPU usage is high from the beginning on the Gui Machine?

[Rob]

Cortex problems can be linked to graphics. Are you displaying anything other than residuals and a few monitors.
Re double precision, without it the data may not be as accurate as you'd like. 50 million particles is also fairly high.

[lei2019]

I am not sure abut GUI machine. Nate from WVU HPC suggested maybe we issue command from this node and this node responsible for the controlling.
I didn't display other things, just residuals and monitors; I use execute command to save pictures. that's all.

[Rob]

Pictures of what?

[lei2019]

The picture shows the GUI interface resource, 1 node, 1 core.

[Rob]

What do you use execute commands to save pictures of?

[Amine Ben Hadj Ali]

Are you able to see similar CPU load if running in batch without graphics?

[lei2019]

I didn't run the case in batch without graphics; I ran it with -gu model one time, it showed the residual windows and monitors. What I can remember is the VNC session was slow; maybe it had the similar cpu load, or it should be normal. Thank you.

[lei2019]

by using top, it show VNC sessions don't consume much Memory;
by using free -g, 38GB is available. So memory should be enough.

[lei2019]

based on cluster administrator, the head node overloaded with too much work; this is the reason why the VNC gui is slow, not because of Fluent.

[Amine Ben Hadj Ali]

Thanks for the feedback. But in general Rob and I presented the steps which we usually start doing / asking when things like that appears! :)
So it will be good if you run one time with -gu -driver null flag and check the CPU consumption on the Cortex.

[lei2019]

Since now I can't stop the simulation, I will try it later. Thank you.