Reply To: AnsysEDT SLURM tight IntelMPI integration issue

[randyk]

Hi,

Please create folder:  $HOME/anstest
Then create file:  $HOME/anstest/job.sh   with the following contents(correct partition and installation path):

#!/bin/bash

#SBATCH -N 3

#SBATCH -n 12

#SBATCH -J AnsysEMTest     # sensible name for the job

#SBATCH -p default         # partition name

 

InstFolder=/opt/AnsysEM/v232/Linux64

JobFolder=$(pwd)

 

# SLURM setup

export ANSYSEM_GENERIC_MPI_WRAPPER=${InstFolder}/schedulers/scripts/utils/slurm_srun_wrapper.sh

export ANSYSEM_COMMON_PREFIX=${InstFolder}/common

srun_cmd="srun --overcommit --export=ALL  -n 1 -N 1 --cpu-bind=none --mem-per-cpu=0 --overlap "

export ANSYSEM_TASKS_PER_NODE="${SLURM_TASKS_PER_NODE}"

 

# Setup Batchoptions

echo "\$begin 'Config'" > ${JobFolder}/${JobName}.options

echo "'HFSS/RemoteSpawnCommand'='scheduler'" >> ${JobFolder}/${JobName}.options

echo "'HFSS 3D Layout Design/RemoteSpawnCommand'='scheduler'" >> ${JobFolder}/${JobName}.options

echo "'HFSS/MPIVersion'='2021'" >> ${JobFolder}/${JobName}.options

echo "'HFSS 3D Layout Design/MPIVersion'='2021'" >> ${JobFolder}/${JobName}.options

echo "'tempdirectory'='/tmp'" >> ${JobFolder}/${JobName}.options

echo "\$end 'Config'" >> ${JobFolder}/${JobName}.options

 

# skip OS/Dependency check

export ANS_IGNOREOS=1

export ANS_NODEPCHECK=1

 

# MPI timeout set to 30min default for cloud suggest lower to 120 or 240 seconds for onprem

export MPI_TIMEOUT_SECONDS=120

 

#copy test project

cp ${InstFolder}/schedulers/diagnostics/Projects/HFSS/OptimTee-DiscreteSweep.aedt ${JobFolder}/OptimTee-DiscreteSweep.aedt

 

# Submit AEDT Job (SLURM requires 'srun' and tight integration change to the slurm_srun_wrapper.sh

${srun_cmd} ${InstFolder}/ansysedt -ng -monitor -waitforlicense -useelectronicsppe=1 -distributed -machinelist numcores=12 -auto -batchoptions ${JobFolder}/${JobName}.options -batchsolve TeeModel:Nominal:Setup1 ${JobFolder}/OptimTee-DiscreteSweep.aedt

$ cd $HOME/anstest/
$ dos2unix $HOME/anstest/job.sh

$ chmod +x $HOME/anstest/job.sh

$ sbatch $HOME/anstest/job.sh 

 

Does this solve successfully, does it distribute across the three machines?

 

thanks

Randy