Learning Forum

[Tomas Llano-Rios]

When running Ansys (2023R1) Mechanical on any node from a cluster, it only acquires the license successfully after the first run has failed. This happened before with fluent, but I fixed it by setting ANSYSLI_TIMEOUT_FLEXLM=20, ANSYSLI_FLEXLM_TIMEOUT_ENV=2000000, ANSYSCL_TIMEOUT_CONNECT=60, and ANSYSCL_TIMEOUT_RESPONSE=300 in ansyslmd.ini (/forum/forums/topic/fluentansys-2023r1-fails-to-acquire-license-on-1st-run-but-succeeds-on-2nd/). Why do these timeout values not work for Mechanical?

Any help you can give me is highly appreciated.

Example command I use to invoke Ansys Mechanical:

ansys231 -b nolist -apip on -p ansys -i remote.dat -o solve.out -dis -machines node01:4 -dir "/path/to/data" –usessh

Here are some relevant logs:

/home/user/.ansys/licdebug.node01.FEAT_ANSYS.231.out

2023/07/18 18:09:02    INFO                Starting Licensing Client Proxy server.

2023/07/18 18:09:02    INFO                /apps/ansys/2023r1/v231/licensingclient/linx64/ansyscl -acl 91922.75297 -nodaemon -log /home/user/.ansys/ansyscl.node01.91922.75297.log

2023/07/18 18:09:02    INFO                Started ANSYSLI server.

2023/07/18 18:11:02    CONNECT_ERROR       ANSYS                                                                                0/0/0/0   91922:FEAT_ANSYS:user@node01.maas:linx64          0:127.0.1.1

                ANSYSLI exited or could not read server port ansyscl.node01.91922.75297.

                Please refer /home/user/.ansys/ansyscl.node01.91922.75297.log for more information.

2023/07/18 18:11:02    CONNECT_ERROR       ANSYS                                                                                0/0/0/0   91922:FEAT_ANSYS:user@node01.maas:linx64          0:127.0.1.1

                ANSYSLI exited or could not read server port ansyscl.node01.91922.75297.

                Please refer /home/user/.ansys/ansyscl.node01.91922.75297.log for more information.

/home/user/.ansys/ansyscl.node01.log

2023/07/18 18:11:03    INFO                ANSYSLI_CMD=/apps/ansys/2023r1/v231/licensingclient/linx64/ansyscl -acl 91922.75297 -nodaemon -log /home/user/.ansys/ansyscl.node01.91922.75297.log

2023/07/18 18:11:03    INFO                ANSYSLI_INITIALIZATION_FILE='/apps/ansys/2023r1/shared_files/licensing/ansyslmd.ini'

2023/07/18 18:11:03    INFO                ANSYSLI_PRODORD_FILE='/apps/ansys/2023r1/shared_files/licensing/prodord/ansysli.prodord.xml'

2023/07/18 18:11:03    INFO                ANSYSLI_TIMEOUT_FLEXLM=20

2023/07/18 18:11:03    INFO                Configuring ACL Core

2023/07/18 18:11:03    INFO                ACL Core Initialized

2023/07/18 18:11:03    INFO                ANSYSCL_PORT=39841

2023/07/18 18:11:03    INFO                Listen Socket Created

2023/07/18 18:11:03    INFO                ANSYSLI_CLIENT_IDLE_TIMEOUT=0

2023/07/18 18:11:03    INFO                ANSYSCL_FNP_PATH=1055@licensing.server

2023/07/18 18:11:03    INFO                Ready to accept connections.

2023/07/18 18:11:03    INFO                ANSYSLI_IP_OVERRIDE option is off.

2023/07/18 18:11:18    INFO                ACL Shutdown

2023/07/18 18:11:19    INFO                Shutting Down ACL Core

2023/07/18 18:11:19    INFO                ACL Core Shutdown

2023/07/18 18:11:19    INFO                License server shutdown is complete. Thank you.

Additionally, If I run ansys231 interactively it shows these errors:

ANSYS LICENSE MANAGER ERROR:

 ANSYSLI exited or could not read server port ansyscl.node01.126749.30360.

 Please refer /home/user/.ansys/ansyscl.node01.126749.30360.log for more information.

 

 

 ANSYS LICENSE MANAGER ERROR:

 Request name mech_2 does not exist in the licensing pool.

 No such feature exists.

 Feature:       mech_2

 License path:  1055@licensing.server:

 FlexNet Licensing error:-5,147

 

 ANSYS LICENSE MANAGER ERROR:

 Request name mech_1 does not exist in the licensing pool.

 No such feature exists.

 Feature:       mech_1

 License path:  1055@licensing.server:

 FlexNet Licensing error:-5,147

 

 ANSYS LICENSE MANAGER ERROR:

 Request name meba does not exist in the licensing pool.

 No such feature exists.

 Feature:       meba

 License path:  1055@licensing.server:

 FlexNet Licensing error:-5,147

 

 Ansys Mechanical Enterprise Academic Research

[MangeshANSYS]

Hello

Does setting

FLEXLM_TIMEOUT=20000000

help ?

[Tomas Llano-Rios]

Hi,

no, it didn't help. I added FLEXLM_TIMEOUT=20000000 to ansyslmd.ini but the issue persists.

[George Karnos]

Hello,

Can you try the following:

Set the following Environment Variable:
FLEXLM_TIMEOUT=60000000

Other possibilities are a slow disk or a security software may not be letting ansyscl startup fast enough.

 

After 2nd try of the binary, any libraries etc might be getting into system cache and getting reused - but this is only my hypothesis based on the behavior reported

[Tomas Llano-Rios]

Hi,

I changed FLEXLM_TIMEOUT to 60000000, but saw no improvement. For the other possibilities you describe:

Slow disk: Since the data was within a NFS export, I did the following:

1. Copied the data (size is 5.2G) to a local SSD: cp -r /home/user/data /tmp/data
2. Changed directory to /tmp/data: cd /tmp/data
3. Launched mapdl from /tmp/data as follows:  mapdl -b nolist -apip on -p ansys -i remote.dat -o solve.out -dis -machines node02:4 -dir "/tmp/data" -usessh

It still failed in the same way. I also tried launching mapdl from a local and remote ansys install (the remote one residing on an NFS export), but both of them have the exact same error.

Security software: The node runs Ubuntu 20.04, but I double checked and UFW is turned off. The coordinador node is the gateway and routes traffic via nftables. I have also tried disabling non-forwarding related rules on nftables, thus allowing all traffic from the licensing server to come in, but it still does not work.

[MangeshANSYS]

Are the compute machines and license server on the same network (same LAN) ?

 

 

[Tomas Llano-Rios]

No. The compute nodes are on a separate LAN. The coordinator (or master) node has two NICS; one in the same network as the license server and another in the same network as the compute nodes. It acts as a router for the compute nodes by forwarding traffic from one LAN to the other via nftables rules.

[MangeshANSYS]

if simply 2 different subets but same LAN then it should be fast. what might be reasons for delays?
Network Latency: any reason the latency might be higher between networks? perhaps heavy network load? routes? multiple DNS ? (try seting environment variable FNP_IP_ENV=1 on compute nodes ?
License Server Resources: Is the license server under heavy load? perhaps also serving other licenses or running other programs?

[Tomas Llano-Rios]

 

The license server IP and hostname are both within /etc/hosts on every node, so lookups are fast. Moreover, I use the license server IP instead of its hostname when setting up the SERVER variable inside ansyslmd.ini. This is how it currently looks:

SERVER=1055@x.x.x.x
ANSYSLI_SERVERS=2325@x.x.x.x
ANSYSLI_FNP_IP_ENV=1
ANSYSLI_TIMEOUT_FLEXLM=20
ANSYSLI_FLEXLM_TIMEOUT_ENV=2000000
ANSYSCL_TIMEOUT_CONNECT=120
ANSYSCL_TIMEOUT_RESPONSE=300
ANSYSLI_TIMEOUT_TCP=300
FLEXLM_TIMEOUT=60000000

The nodes can reach the license server fairly quick:

Coordinator node (TCP test):

traceroute -T x.x.x.x

traceroute to x.x.x.x (x.x.x.x), 30 hops max, 60 byte packets
 1  license.server (x.x.x.x)  0.274 ms  0.210 ms  0.239 ms

node02 (TCP test):

traceroute -T x.x.x.x

traceroute to x.x.x.x (x.x.x.x), 30 hops max, 60 byte packets
 1  node00.maas (y.y.y.y)  0.275 ms  0.229 ms  0.213 ms
 2  license.server (x.x.x.x)  0.393 ms  0.418 ms  0.404 ms

Coordinator node (ICMP test):

PING x.x.x.x (x.x.x.x) 56(84) bytes of data.
64 bytes from x.x.x.x: icmp_seq=1 ttl=127 time=0.564 ms
64 bytes from x.x.x.x: icmp_seq=2 ttl=127 time=0.562 ms
64 bytes from x.x.x.x: icmp_seq=3 ttl=127 time=0.534 ms

--- x.x.x.x ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2043ms
rtt min/avg/max/mdev = 0.534/0.553/0.564/0.013 ms

node02 (ICMP test):

PING x.x.x.x (x.x.x.x) 56(84) bytes of data.
64 bytes from x.x.x.x: icmp_seq=1 ttl=128 time=0.314 ms
64 bytes from x.x.x.x: icmp_seq=2 ttl=128 time=0.298 ms
64 bytes from x.x.x.x: icmp_seq=3 ttl=128 time=0.320 ms

--- x.x.x.x ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2052ms
rtt min/avg/max/mdev = 0.298/0.310/0.320/0.009 ms

 

I tried setting FNP_IP_ENV=1 when I executed mapdl, but the issue persists. From the command line:

FNP_IP_ENV=1 mapdl -b nolist -apip on -p ansys -i remote.dat -o solve.out -dis -machines node02:4 -dir “/tmp/zdusa2sc.s1v” -usessh

As for the license server, It serves licenses to multiple machines, but it is not under heavy load. When I run mapdl from the coordinator node there are not issues. It acquires the license on the first try and runs fine every time (when this happened with fluent in the past I noticed the same behaviour; the coordinator would acquire the license without issues, but the compute nodes would take more time to the point I had to set longer timeouts for FLEXLM and ANSYSCL in ansyslmd.ini).