Learning Forum

[whitee8]

Hello,

I have been attempting to use the new Ansys Rocky MPI support (2026R1) to parallelize a simulation across multiple nodes. However, it has proven challenging to set up correctly; there are no official examples and the command line flags are not terribly helpful either.

I am launching Rocky using slurm, which appears to be a supported scheduler. As an example call within an interactive session across two nodes:

srun -N2 --ntasks-per-node=1 

    $ROCKY --headless --simulate "$SIM_PROJECT"

    --resume=1

    --mpi-ncpus-per-process 4

    --use-mpi

    > rocky_mpi_test.log 2>&1

In the log file:

json:{"status": "SIMULATING", "message": "full_sim.rocky"}

json:{"status": "SIMULATING", "message": "full_sim.rocky"}

QSocketNotifier: Can only be used with threads started with QThread

lsblk: rootfs: not a block device

json:{"status": "INITIALIZING", "progress": 0.0}

json:{"status": "INITIALIZING", "message": "Initializing"}

json:{"status": "SIMULATING", "message": "full_sim.rocky"}

json:{"status": "SIMULATING", "message": "full_sim.rocky"}

json:{"status": "ERROR", "message": "Unexpected dialog boxninstance=MessageBox.Criticalnargs=('Project locked',)nkwargs={}nnSetup errors:nnnThe Project you're trying to simulate is Locked by machine with hostname ___ by the process with pid 1593676nnIf you're sure no one else is working on this project delete the file "____.rocky.lock""}

 

The simulation starts on both nodes, but one node seems to trip over the lock file produced by the other. I imagine that there must be something else actually causing the issue?

 

The simulation itself runs fine when using a single node, so I dont think that is the issue. I appreciate any help on this issue, and realize that it is in beta so errors are to be expected. Let me know if more information is required (the simulation.rocky20.log files don't have any glaring errors either). If its useful this is on the Purdue Anvil HPC with a InfiniBand interconnect using openmpi.

 

Thank you!