Learning Forum

[j-garmatter]

Hello,

I installed Ansys Fluent on one compute node in my Linux cluster. This is a silent install, no GUI. I was able to run my test case on this node and through SLURM from this node.

This was the command I used: fluent 3ddp -g -i test_input.txt -t4 -scheduler=slurm -scheduler_queue=Fluent -p=eth

However, I'd like to submit the job to the SLURM controller/head node. This is the command I used: fluent 3ddp -g -i test_input.txt -t4 -scheduler=slurm -scheduler_headnode=myHeadNode -scheduler_queue=Fluent -p=eth

I got these errors:

1. I had to enter my account's password three times to submit the job the head node.
2. The job timed out after 10 minutes or so.
   1. Luckily I found this environment variable to force the job to timeout sooner: FLUENT_START_COMPUTE_NODE_TIME_OUT=30
3. Once the job timed out I saw the following:

   1. My transaction file:
      Connected License Server List:  
      Unexpected license problem; exiting.

       

   2. My standard error file:
      Unexpected license problem; exiting.

Does anyone know why the job would run without issues on the host that has Fluent installed but have so many issues on the head node? Do I need to install some Ansys component on the node with the SLURM controller?

[j-garmatter]

Perhaps I should clarify a few things about my cluster and understanding.

I have a three node cluster, two of which are compute nodes and one is the head node. The head node has the SLURM controller while the compute nodes have the regular SLURM process running. There is shared storage between all three hosts. Users log in to the head node, move files to the shared storage, then submit their jobs to SLURM. After that, SLURM runs the job on the appropriate compute node and the output is written to shared storage where the user can view it.

When I run the job from the compute node that I installed Fluent on, the job runs successfully, no license errors are thrown. I can also submit the job to SLURM from the compute node, as noted in my initial post, and it runs successfully too, no license errors are thrown. This method is not ideal because the end users don't have direct access to the compute nodes. They must submit their jobs through the head node. If I specify the SLURM head node with "-scheduler_headhnode", as I did in my initial post, I get the "unexpected license error". 

My end goal is that users may submit a Fluent job through the head node from their own machines with Fluent installed. This allows the user to utilize the OS and GUI from their own machine (that they are used to, instead of requiring them to understand Linux commands) to submit their jobs to the cluster. Being able to run Fluent through the head node via CLI is merely an intermediate step to get to my end goal.

Do I need to install an Ansys component on the SLURM head node to allow proper license allocation through it?

[j-garmatter]

I figured out the solution. I had to add my /etc/profile.d/fluent.sh file, which contains environment variables for fluent, to the SLURM head node.

I believe the most important variable in that file is "ANSYSLMD_LICENSE_FILE=1055@mylicensehost" and possibly "ANSYS_LIC_UTILITY=/usr/ansys_inc/shared_files/licensing/lic_admin".

I haven't seen the "Unexpected license problem" since I added the fluent.sh script. My jobs ran successfully!

This solution is strange to me. The SLURM head node doesn't run any aspect of fluent so it doesn't make sense to me why this must be added to the environment. Unless the environment variables are passed along to the compute node...

[MangeshANSYS]

Hello

that is correct, license settings and network connections need to be configured so the application can connect to license server from the computer (node) on which the application starts.

If you are all set with the issue then please mark this thread as answered ?

[j-garmatter]

Mangesh,

Why does the head node, which doesn't have fluent or another ansys component installed, need to know where the ansys license server is?

The head node just submits the job script to the compute node. The compute node already had its variables declared in an /etc/profile.d/ script.

I don't understand why the license variables need to be specified from the node that doesn't run Fluent.

[MangeshANSYS]

linux environments usually share same settings and file systems (NFS)

as mentioned earlier, the machine where application is launched needs to have license server information configuired and a network connection to the license server

hope this helps

[j-garmatter]

All I can say is that I did have the license info configured on the machine where the app was launched (the compute node).

The app couldn't be launched from the head node as fluent isn't installed on the head node.

Despite this, I still had to add the license settings to the head node. I included this info in case it helps anyone else.

Mangesh, this post was marked as answered on the 11th. Thank you for your time.