Learning Forum

[mingdian]

Hi. I am using AnsysEM 19.1 (linux system) on HPC for parameter sweeping (see Figure 1). The project includes 1680 cases and will take nearly 2 days to complete. However, the HPC has a time limit (at most 1 day). I wonder if there is any method I can accelerate the progress and shorten the time into 1 day? I list my questions as following:

1. Do you have any advice on running AnsysEM 19.1 on HPC? To accelerate the parameter sweeping progress?
2. I have updated Number of Processors (see Figure 2). It seems it has some help to the simulation. Is the number of processors should be the number of CPU core of the HPC machine I allocate?
3. I also changed Analysis Configuration (see Figure 3 and Figure 4)and updated the core number to be the core number of HPC machine I am using. Does it help the simulation?
4. I should set HPC License be Pool, right?
5. How could I use GPU for Ansys (Enable GPU in Figure 5)? Will it be helpful to speed up the simulation progress?
6. Is there a checkpoint and restart function on AnsysEM 19.1? For example, I can pause parameter sweeping, save the progress and close the software. Later then, I can load the checkpoint and restart the parameter sweeping.
7. Is there any method that I can monitor CPU/Memory usage during running the software?

That's all my questions. Thank you!

Figure 1

[AndyJP]

1. That depends on HPC you have.
Normally, do not give more than 6 threads per worker(task). At least in the code ver 13, and 2015 I had no improvements above 6 cores per task, 4 cores is optimal.
Try not using multiprocessor machines with shared memory between multiple tasks. Good rule is 1 task : 1 machine with its own RAM.
If you have a choice, get less cores per machine (intel 6-8 core CPUs) with higher top clocks per each CPU.
When you set number of workers equal or less to number of machines, HFSS will distribute every sweeping step, one per machine, and you will have optimal performance.
2.HFSS thinks of cores as of CPUs and not capable of analyzing CPU and NUMA layout. That's why HyperThreading must be disabled. BTW, I don't like "Automatic" distribution; prefer to set configuration exactly. And Automatic disables GPU acceleration.
3. What do you mean by "core number of HPC machine"? I thought, you have a cluster, or a blade server. Yes, the whole number of cores you plan to use. For uniform distribution it is {core number}={cores per task}*{tasks=machines count}.
4. That depends on what you have. Pool has "nonlinear" number of available cores per number of HPC licenses; adding one gives more cores. 1 HPC=4 cores, 3 HPC = not 12, but 16 cores, and so on. But only for one problem/workplace. Per-core licenses are eache per one core. But these can be shared freely between designers.
5. When you disable Automatic Settings, and enable GPU in options. GPU acceleration works only in transient with pure dielectrics and metals in older versions. I do not remember how it is in 2019, but in ver.2020 GPU works with DrivenModal. But again, only for pure dielectrics. If you have any kind of anisotropy, it falls back to CPU.
Geforce boards aren't supported in 2020R1 and newer. In older versions they worked. But not sure about your version.
6. Yes. The results used in report plots are saved per every iteration. Fields are saved for the central (meshing) frequency; but you can enable saving fields for every simulation step.
When HFSS sees results for the iteration requested, it skips the iteration. But if you change parameters in between, it will start from zero.
So you can stop, and continue another day. But do not change any setup in between.
The solver itself can not be interrupted without loss of data for the current iteration.
7. It is OS related question. Ask the OS community, support, or your system administrator.

Edit: Array . Array I've just unstuck this from the moderation queue.

[mingdian]

Hi Andy,
Thanks for your detailed answer!