{"id":199657,"date":"2022-01-11T13:46:37","date_gmt":"2022-01-11T13:46:37","guid":{"rendered":"\/forum\/forums\/reply\/199657\/"},"modified":"2022-01-11T13:47:29","modified_gmt":"2022-01-11T13:47:29","slug":"199657","status":"publish","type":"reply","link":"https:\/\/innovationspace.ansys.com\/forum\/forums\/reply\/199657\/","title":{"rendered":"Reply To: (MPI?) problem when running System Coupling on Linux HPC cluster"},"content":{"rendered":"

Hi Judith
\nOk, that’s interesting that you say it worked when 1 core was assigned to each solver.
\nrunFSI should be a python script, better with extension .py. I haven’t tested whether extension .txt will be interpreted as a python script. Likely though if it works with 1 core per solver this is not a problem.
\nSince it seems to be happy with 1 core each but not more, I think we should look at the job submission method and core assignment. Have you got a SLURM submission script that sets environmental variables? Here is an example you can use for SyC jobs on SLURM:
\n#!\/bin\/bash -l<\/strong>
\n#<\/strong>
\n# Set slurm options as needed<\/strong>
\n#<\/strong>
\n#SBATCH –job-name SYSC<\/strong>
\n#SBATCH –nodes=2<\/strong>
\n#SBATCH –partition=ottc02<\/strong>
\n#SBATCH –ntasks-per-node=32<\/strong>
\n#SBATCH –output=%x-%j.out<\/strong>
\n#SBATCH –error=%x-%j.err<\/strong>
\n#SBATCH –export=ALL<\/strong>
\nexport AWP_ROOT212=\/sw\/arch\/Centos8\/EB_production\/2021\/software\/ANSYS\/2021R2\/v212<\/strong>
\n#<\/strong>
\nexport SYSC_ROOT=${AWP_ROOT212}\/SystemCoupling<\/strong>
\n#<\/strong>
\n# print job start time and Slurm job resources<\/strong>
\n#<\/strong>
\ndate<\/strong>
\necho “SLURM_JOB_ID : “$SLURM_JOB_ID<\/strong>
\necho “SLURM_JOB_NODELIST : “$SLURM_JOB_NODELIST<\/strong>
\necho “SLURM_JOB_NUM_NODES : “$SLURM_JOB_NUM_NODES<\/strong>
\necho “SLURM_NODELIST : “$SLURM_NODELIST<\/strong>
\necho “SLURM_NTASKS : “$SLURM_NTASKS<\/strong>
\necho “SLURM_TASKS_PER_NODE : “$SLURM_TASKS_PER_NODE<\/strong>
\necho “working directory : “$SLURM_SUBMIT_DIR<\/strong>
\n#<\/strong>
\necho “Running System Coupling”<\/strong>
\necho “System coupling main execution host is $HOSTNAME”<\/strong>
\necho “Current working directory is $PWD”<\/strong>
\n#echo “ANSYS install root is $AWP_ROOT212”<\/strong>
\necho “System coupling root is $SYSC_ROOT”<\/strong>
\necho “Run script is $1”<\/strong>
\necho<\/strong>
\n“$SYSC_ROOT\/bin\/systemcoupling” -R runFSI.txt<\/strong>
\nyou’d have to change job name, nodes, partition and ntasks-per-node.
\nPaul<\/p>\n","protected":false},"template":"","class_list":["post-199657","reply","type-reply","status-publish","hentry"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t