Learning Forum

[CMCsupport1]

One of our student users has been running a simulation on her laptop prior to loading it into a ‘Compute Canada’ HPC node.

Her laptop is running Windows 10, 64-bit level, and her ANSYS installation is 19.2. We supply her with her licence. According to the ANSYS "Academic Product Features" table, she should have a built-in capability to run her simulation with up to 16 cores, after which she should be able to pull out additional HPC licence features to increase the number of cores for her simulation work. She reports that she seems to be stuck using only four cores.

Do you have suggestions we could pass along to this student?  Full problem description will be attached.  

[tsiriaks]

Hi greigp,

Ansys policy doesn't allow Ansys employee to download files from the forum, especially those that are not image files. Could you post the screenshot of them in the content ?

Do you have any commercial license on the license server ? 4 built-in HPC seems like those commercial licenses. All academic licenses do offer 16 built-in HPC like you mentioned.

Thanks,

Win

[AmyTheSun]

Hello greigp,

Thanks so much for taking the extra mile and post the question here. 

Hello Win, 

Thanks for your prompt reply. I am not sure the license type. That would be best answered by @greigp. I believe this should be an academic license. I tried to set up HPC for 16 cores in my local Windows machine and got no warning messages. (Of course, I only have 4 cores on my laptop) 

The issue we are having right now is to use the proper commands to specify HPC options running on a Linux machine. 

We have tried the following:

-batchoptions 'HFSS/NumCores'='16’ -batchoptions 'HFSS/HPCLicenseType'='pool'

and

-batchoptions 'HFSS/NumCoresPerDistributedTask'='16' -batchoptions 'HFSS/HPCLicenseType'='pool'

None of the above commands works. The HFSS log file still indicates that 4 cpu cores were used. 

 

I attached a screenshot of the document attached by greigp for easier viewing. Please note I am running on AnsysEM19.1. 

 

[tsiriaks]

Hi AmyTheSun,

I see. Thanks for the additional info. Now, I know that I misunderstood the issue. I thought you tried running 16 cores and got the licensing error saying you can run only 4. If it always default you to run only 4 cores, this is the matter of simulation setting, not anything about license server or licensing.

Now the workflow does matter. How are you submitting the job to this SLURM cluster ?

1. setting up RSM , then use the AnsysEM GUI on your Windows machine to submit the job on the cluster

2. remote login to one of the cluster nodes, then use batch submission to submit the job

3. remote login to one of the cluster nodes, then submit the job via AnsysEM GUI from that node

Thanks,

Win

[AmyTheSun]

Hello Win, 

I submit the job using Option 2. During debugging, we were able to see that 16 cpu cores were obtained but the HFSS simulation was only running on 4 cores. 

 I have attached a screenshot of the script used for job submission. 

[CMCsupport1]

Hi greigp,

Ansys policy doesn't allow Ansys employee to download files from the forum, especially those that are not image files. Could you post the screenshot of them in the content ?

Do you have any commercial license on the license server ? 4 built-in HPC seems like those commercial licenses. All academic licenses do offer 16 built-in HPC like you mentioned.

Thanks,

Win

 

Good afternoon, Doctor Siriaksorn,

Here is the contents of the original *.pdf file attached to the opening remarks:

 

-&-&-&-&-&-&-

One of our student users has been running a simulation on her laptop prior to loading it into a ‘Compute Canada’ node.

Her laptop is running Windows 10, 64-bit level, and her ANSYS installation is 19.2. We supply her with her licence. Our ANSYS subscription is for a ‘Campus Solutions’ bundle. According to this document:

https://www.ansys.com/-/media/Ansys/corporate/files/pdf/product/academic/academic-products-features-table.pdf?la=en&la=en

… she should have a built-in capability to run her simulation with up to 16 cores (licence feature hfss_desktop? hfss_gui? hfss_something) after which she should be able to pull out additional HPC licence features (hfsshpc) to increase the number of cores for her simulation work, to the limit of the hardware or of the feature number available.

Using our own diagnostic tools, we know that this student can reach our licence properly. Our debug logs show there is no firewall or connection issues between her laptop, the ‘Compute Canada’ node and our ANSYS licence server. We have no internal restriction blocking her use of the HPC features (e.g. an options file). We also know that student must set her ‘HPC Licence’ to ‘Pool’:

ANSYS AEDT Desktop -> Tools -> Options -> HPC and Analysis Options -> Options Tab ->

She reports that she seems to be stuck using only four (4) cores. In other cases listed in your ANSYS Student Community, it would seem as though users have this problem if they are not using the ANSYS Academic software, or if the server licence manager must be upgraded. The latter problem is not applicable, as other students are not suffering this trouble.

The node on which she is trying to run her simulation is ‘Cedar’, and the job scheduler running on Cedar is SLURM. She is getting help with the SLURM configuration from ‘Compute Canada’ staff. The core restriction isn’t a SLURM problem, though.

https://docs.computecanada.ca/wiki/Cedar

Do you have other suggestions we could pass along to this student? We will check with this woman to ensure she is using a copy of your Academic CAD suite.

Thank you,
CMC Microsystems support staff

 

[tsiriaks]

Hi greigp,

Thank you for the info.

Hi AmyTheSun,

I don't think that batch option is used to specify the total cores for the solve, it only/manually defines how many cores to use per task. I will have to ask others to look into this tomorrow since I'm not very familiar with batch submission command.

In any case, I do have below batch submission template, you can try play around with this in the meantime.

 ${InstFolder}/ansysedt -distributed -machinelist numcores=${AutoTotalCores} -auto -monitor -ng -batchoptions ${JobFolder}/${JobName}.options -batchsolve ${AnalysisSetup} ${JobFolder}/${Project} > ${JobFolder}/${JobName}.progress

 

This ${AutoTotalCores}  would be 16 in your case and you can exclude   -batchoptions ${JobFolder}/${JobName}.options

Thanks,

Win

[tsiriaks]

Note: This thread has been escalated since SLURM scheduler needs some customization before it can properly work with AnsysEM

[AmyTheSun]

Hello Win, 

After talking to Compute Canada staff, I realized I failed to mention that we are not trying to run distributed simulation. We want to run the simulation on a single node, which usually contains 32 cores or up. That being said, we need the multicore functionality from ansysedt command line. 

Best,

Amy 

[tsiriaks]

Hello Amy,

Thanks for the additional information. 

Even with a single node simulation, there are some customization required for SLURM to work properly with AEDT.

We are working directly with Compute Canada staff.

Thanks,

Win

[AmyTheSun]

Hello Win, 

Thank you very much for your prompt reply. I look forward to hearing from you soon. 

Best,

Amy