{"id":352244,"date":"2024-02-19T09:10:38","date_gmt":"2024-02-19T09:10:38","guid":{"rendered":"\/forum\/forums\/topic\/running-a-test-job-on-a-slurm-cluster\/"},"modified":"2024-07-03T07:57:01","modified_gmt":"2024-07-03T07:57:01","slug":"running-a-test-job-on-a-slurm-cluster","status":"closed","type":"topic","link":"https:\/\/innovationspace.ansys.com\/forum\/forums\/topic\/running-a-test-job-on-a-slurm-cluster\/","title":{"rendered":"Running a test job on a SLURM Cluster"},"content":{"rendered":"

This guide will help you validate that the cluster resources have been deployed successfully on Ansys Gateway.
Please follow the recommended HPC cluster configurations by application in the documentation page to setup your resources: Recommended Configurations by Application (ansys.com)<\/a>
Cluster Requirements: <\/p>\n

To configure an HPC workflow that uses a Slurm cluster the following resources are required in the following order : <\/p>\n

 <\/p>\n\n\n\n
\n

 <\/span><\/p>\n

    \n
  1. NFS file storage server with a shared folder.<\/span><\/strong><\/a><\/li>\n
  2. Linux Virtual Desktop with Slurm Controller and KDE<\/span><\/strong><\/a><\/li>\n
  3. HPC Cluster with Slurm Node with NFS and EFA<\/span><\/strong><\/a><\/li>\n<\/ol>\n<\/td>\n
\n

<\/a>
<\/span><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Please ensure that all resources are ready and RUNNING<\/strong> before proceeding.<\/p>\n\n\n\n\n
\n

<\/a>
<\/span><\/p>\n<\/td>\n

\n

<\/a>
<\/span><\/p>\n<\/td>\n<\/tr>\n

\n

<\/a>
<\/span><\/p>\n<\/td>\n

\n

 <\/span><\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Now we can proceed with the execution of the test script.<\/p>\n

    \n
  1. Take a note of the shared folder name
    <\/a><\/li>\n
  2. Connect to the Slurm Controller VM. (use RDP, or SSH)<\/li>\n
  3. Create a file “test.sh” and paste the contents below:\n
    #!\/bin\/bash
    export NFSDIR=\/mnt\/$1
    if [ ! -d \"$NFSDIR\" ]; then
      echo \"${NFSDIR} does not exist.\"
      exit 1
    fi
    mkdir -p ${NFSDIR}\/testjob
    NODES=$(sinfo -p ACS_cluster -o \"%a %F\" | tail -n1 | grep \"up\" | awk -F'\/' '{print $4}')
    if [ -z \"${NODES}\" ]; then
      echo \"No available nodes on ACS_cluster partition\"
      exit 1
    fi
    sbatch -N$NODES << 'EOF'
    #!\/bin\/bash
    #SBATCH --job-name=\"TestJob\"
    #SBATCH --exclusive
    #SBATCH --output=\"%x-%j\".out
    #SBATCH --error=\"%x-%j\".err
    #SBATCH --partition=ACS_cluster
    date
    echo \"------------\"
    echo \"SLURM_JOB_ID : \"$SLURM_JOB_ID
    echo \"SLURM_JOB_NODELIST : \"$SLURM_JOB_NODELIST
    echo \"SLURM_JOB_NUM_NODES : \"$SLURM_JOB_NUM_NODES
    echo \"SLURM_NODELIST : \"$SLURM_NODELIST
    echo \"SLURM_TASKS_PER_NODE : \"$SLURM_TASKS_PER_NODE
    echo \"WORKING DIRECTORY : \"$SLURM_SUBMIT_DIR
    echo \"NFS STORAGE DIRECTORY: \"$NFSDIR
    echo \"------------\"
    cd $NFSDIR\/testjob
    touch TestJob-$SLURM_JOB_ID.txt
    MACHINELIST=$(srun hostname)
    for HOST in ${MACHINELIST}; do
            srun --nodes=1 echo \"hello from node ${HOST}\" >> TestJob-$SLURM_JOB_ID.txt &
    done
    #Wait for sub-processes to finish
    wait
    EOF<\/pre>\n<\/li>\n
  4. Run “test.sh” script, you need to provide the shared folder name from step (1)\n
    bash test.sh sharedfolder<\/pre>\n<\/li>\n
  5. Check the output, it should look like this\n
    cat \/mnt\/sharedfolder\/TestJob-1*<\/pre>\n
    <\/a><\/li>\n<\/ol>\n

    References<\/strong>:     Testing a Slurm Cluster (ansys.com)<\/a><\/pre>\n","protected":false},"template":"","class_list":["post-352244","topic","type-topic","status-closed","hentry"],"aioseo_notices":[],"acf":[],"custom_fields":[{"0":{"_bbp_author_ip":["23.206.193.146"]," _bbp_last_reply_id":["0"]," _bbp_likes_count":["0"],"_btv_view_count":["1952"],"_bbp_topic_status":["answered"],"_bbp_status":["publish"],"_edit_lock":["1719992930:20790"],"_bbp_likes_count":["1"],"_bbp_topic_id":["352244"],"_bbp_forum_id":["233598"],"_bbp_engagement":["20790"],"_bbp_voice_count":["1"],"_bbp_reply_count":["0"],"_bbp_last_reply_id":["0"],"_bbp_last_active_id":["352244"],"_bbp_last_active_time":["2024-02-19 09:10:38"]},"test":"nikos-nikoloutsakosansys-com"}],"_links":{"self":[{"href":"https:\/\/innovationspace.ansys.com\/forum\/wp-json\/wp\/v2\/topics\/352244","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/innovationspace.ansys.com\/forum\/wp-json\/wp\/v2\/topics"}],"about":[{"href":"https:\/\/innovationspace.ansys.com\/forum\/wp-json\/wp\/v2\/types\/topic"}],"version-history":[{"count":2,"href":"https:\/\/innovationspace.ansys.com\/forum\/wp-json\/wp\/v2\/topics\/352244\/revisions"}],"predecessor-version":[{"id":370268,"href":"https:\/\/innovationspace.ansys.com\/forum\/wp-json\/wp\/v2\/topics\/352244\/revisions\/370268"}],"wp:attachment":[{"href":"https:\/\/innovationspace.ansys.com\/forum\/wp-json\/wp\/v2\/media?parent=352244"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}