Learning Forum

[Martin Cuma]

Hello, I am doing user support at an University research computing center, and working with an user who has trouble getting System Coupling to work.

This student enters the two systems she's trying to simulate into the System Coupling GUI (Fluent and Mechanical), it all looks OK but when she pushes the "solve" option, nothing happens and the lower status bar is saying "Not Connected".

I installed Anysys 21.1 as instructed, and set up LMod environmental module to load the Ansys environment. All the Ansys components are found from the command line, and work fine standalone. I am not seeing any specific instructions in the Install Guide for setting up the System Coupling, so, I would have assumed that the coupling gets set up during installation, but, since it does not work I suspect that's not the case.

I would appreciate if someone could point me to how can I check that the System Coupling is set up correctly, and/or, how can I tell System Coupling where the components are.

Thanks,

Martin

[Konstantin]

This error is most commonly caused by a failure to obtain a license. In particular for academic customer license sharing must be disabled in the user license preferences. From Workbench select "Tools > License Preference" and select "User a separate license for each application". Note that a System Coupling simulation using Fluent and Mechanical will use two license tasks while solving.

[Konstantin]

Depending on specific modeling need of the end user, the built in FSI capability in Fluent can be used. It's new in 2021 and has several limitation, one of which in the assumption of elastic deformations. This FSI is entirely contained in Fluent and doesn't require a connection to Ansys Mechanical.

[Martin Cuma]

Hi Keaton thanks for the reply. I am not seeing Tools - License Preferences in my Workbench, 2021 R1 on Linux started with "runwb2". Did this move somewhere else?
I do see it in 2020 R1, will tell the researcher to try the 2020 R1, but, it'd be good to know where this is in the 2021 R1 since she wants to use capabilities in that release.
Thanks, Martin

[Konstantin]

indeed, License Preferences no longer exists in 2021 due to licensing changes that made it redundant. With 2021 for FSI Mech & Fluids will use separate licenses due to the change in licensing. Mech will use Mech licensing and Fluids will use the CFD features. What you likely need is to connect to the Licensing Portal and download an updated license with the new features. Please refer to:
/forum/discussion/28989/ansys-licensing-updates-for-academic-2021-r1#latest

[Martin Cuma]

Hi Keaton with the 2021, the System Coupling get stuck in the "Awaiting connections from the coupling participants" stage. The logs are cycling:
CTrace (0): Entering serveOneRequest[PhysicsCoupling/AnsRpcWrapper/src/AnsRpcServer.cpp@62]
Client has disconnected
CTrace (0): Leaving serveOneRequest (0.07 s)
I do see a bunch of the System Coupling and Fluent processes running, and cfd_base, cfd_solve_level1 and cfd_solve_level2 license features checked out at the license server.
I contacted the group that runs the license server and they say they did update the 2021 license. lmstat is saying it's running "ansyslmd: UP v11.17.2". Is that the new license that you were mentioning?
I also noticed that in file sycGui.srv in the SC directory, the IP of the machine I run on is listed as 50373@155.101.16.75. I presume that's the port over which the SC components communicate. Is there any firewall requirement for that port? It's a high port so it should be open but I want to make sure.
Thanks Martin

[Stephen Orlando]

Hi Martin Lets come back to the licensing. I want to check on a few other things. Please let me know the following.
Is the student using one of the tutorial or their own simulation? It would be best to use one of the tutorials to debug this to ensure that we know we're working with simulation that runs. Here is one of the tutorials: https://ansyshelp.ansys.com/account/secured?returnurl=/Views/Secured/corp/v211/en/sysc_tut/sysc_tut_reedvalve_fluent.html
Are they running on a local machine or on a cluster?
If running on a local machine, are the files on a local drive or are they on a remote drive, or OneDrive?
Are there any error or warning messages in the output files for each solver? For Fluent, look at the transcript .trn file. For Mechanical, look at the .out file. Please copy and past any error messages to your reply.
Steve

[Martin Cuma]

Thanks Steve, I am in the process of setting up a call with the student so we'll go over your suggestions. This does run on a cluster with NFS mounted file servers (= all files visible on all cluster nodes). The tutorial is a good suggestion, we've been using her own simulation.

[Stephen Orlando]

Good to know it's on a cluster. Does the cluster have a job scheduler or do the users have unrestricted access to the compute nodes?

[Martin Cuma]

Hi everyone, thanks again for all the feedback. It took us a little while to run through the various iterations of running the ReedValve example, but, I think now I have a decent idea of what's going on.
I can run the ReedValve System Coupling example both on my Linux desktop, and on the cluster interactive nodes - both of these machines have public IP and direct Internet access - to go to our College of Engineering (COE) IP space where the license server is. But, with the cluster compute nodes, I can start System Coupling, choose directory, it does then let me define all the coupling parameters, but, when I hit "solve", the status immediately goes to "Exited". The verbose log does not provide much of info, just repeating:
ProvisionalCCL::ProvisionalCCL root /SystemCoupling/CouplingParticipant:FLUENT-1
BoolFromEnvironment: CUE_PHYSVAL_DONTUSECACHE is off
CallbackDependencyAgent::doExecute
In CallbackDependencyAgent::doExecute, exprParams:
The cluster compute nodes are on a private network so they need to NAT to the outside. Our networking staff made some adjustments to the NAT to be able to communicate with the COE license server, which means that we can run all the Ansys applications on the compute nodes. But, I suspect that the System Coupling needs to have some other network port(s) open to the license server in order to function.
Do you know if there are some additional network/port requirements in the communication with the license server that the System Coupling requires?
Thanks.

[Stephen Orlando]

Hi Martin,
A few more clarifying questions before discussing the ports.
1, "Our networking staff made some adjustments to the NAT to be able to communicate with the COE license server, which means that we can run all the Ansys applications on the compute nodes."
Does this mean that you've successfully run Fluent on the compute nodes? If so, what is the execution command used to start a Fluent run on the compute nodes? I want to understand how the specific names of the compute nodes are given to Fluent.
2, Is a job scheduler used to access the compute nodes? ie SLURM, PBS, LSF, or UGE?

[Martin Cuma]

Hi Steve, thanks for writing back.
We are using SLURM for scheduler, and now I am testing the System Coupling only on a single compute node so I don't use any host list to run the simulations, just the localhost radio button.
I start fluent with the plan "fluent" command, and then choose the case and localhost for the local execution. Same thing with the Workbench, just run "runwb2".
I did also verify that the Fluent works in multi-node fashion by hand filling the host information. Speaking of that (I have another question open on this) - can Fluent auto-find that it's inside of a SLURM job and fill in the host list automatically?
Thanks.

[Stephen Orlando]

Ok, we're making progress. System Coupling supports the SLURM job scheduler (as well as PBS, LSF, and UGE). Sending a System Coupling job to a job scheduler requires launching System Coupling from the Command Line. Instructions for this are outlined in this discussion: /forum/discussion/comment/117355#Comment_117355
Fluent supports SLURM as of 2021R2 which is out now. Please see the Fluent documentation for more details.
Steve

[Martin Cuma]

Hi Steve,
I think we're starting to get somewhere. I ran the CLI example, works fine on the interactive node. On the compute node, within a SLURM job it crashes with the attached output sc_compnode_slurm.txt. Looks like it's complaining on the SLURM host list.
When I run outside of SLURM (direct ssh to the node), it goes a bit farther but it crashes trying to launch the Mechanical (sc_compnode_noslurm.txt)
For the within the SLURM run - how do I feed in the host list?
And for outside the SLURM - which I figure we'd get too if we get past the host list - any idea on that?
Thanks, Martin

[Stephen Orlando]

Hi Martin Ansys employees aren't able to download files from the forum. Please copy the relevant error messages into the post. System Coupling will automatically get the host list in the background so you don't need to specify it manually. A few more questions:
Which version of Ansys are you using?
Can you copy the Slurm submission script into the Post, so that I can see how the job is submitted

[Martin Cuma]

OK. sorry about that. We're using Ansys 2020.1 now, and I run interactive job with salloc allocating the node, so, there's no Slurm script
notchpeak1 $ salloc -t 4:00:00 -p notchpeak-shared-short -A notchpeak-shared-short -N 1 -n 8
then once I get the terminal back on the compute node, I just cd to the directory where the input files are and run
notch308 $ systemcoupling -R run.py
Here's the output of that, inside of the Slurm interactive job:
$ systemcoupling -R run.py

ANSYS(R) System Coupling

Executing from: /uufs/chpc.utah.edu/sys/installdir/ansys/20.1/v201/SystemCoupling/bin/systemcoupling

2020 R1

Point Releases and Patches installed:

ANSYS, Inc. Products 2020 R1
Autodyn 2020 R1
LS-DYNA 2020 R1
CFX (includes CFD-Post) 2020 R1
Chemkin 2020 R1
EnSight 2020 R1
FENSAP-ICE 2020 R1
Fluent (includes CFD-Post) 2020 R1
Forte 2020 R1
Polyflow (includes CFD-Post) 2020 R1
TurboGrid 2020 R1
ICEM CFD 2020 R1
Aqwa 2020 R1
Customization Files for User Programmable Features 2020 R1
Mechanical Products 2020 R1
Icepak (includes CFD-Post) 2020 R1
Remote Solve Manager Standalone Services 2020 R1
ACIS Geometry Interface 2020 R1
Catia, Version 5 Geometry Interface 2020 R1
NX Geometry Interface 2020 R1
Parasolid Geometry Interface 2020 R1
ANSYS, Inc. License Manager 2020 R1

(c) 2014-2020 ANSYS, Inc. All rights reserved. Unauthorized use, distribution
or duplication is prohibited. This product is subject to U.S. laws governing
export and re-export. For full Legal Notice, see documentation.

executing script 'run.py'

+-----------------------------------------------------------------------------+
| The most recent execution of the LoadParticipants command reset the|
| datamodel.All content has been cleared.|
+-----------------------------------------------------------------------------+
Caught exception in ApplicationContext.mapModules
Traceback (most recent call last):
File "PyLib/kernel/framework/ApplicationContext.py", line 152, in __call__
File "PyLib/kernel/framework/ApplicationContext.py", line 61, in _makeComponent
File "PyLib/kernel/framework/ApplicationContext.py", line 59, in
File "PyLib/kernel/framework/Annotate.py", line 50, in injected
File "PyLib/kernel/remote/service/__init__.py", line 244, in __init__
File "PyLib/kernel/remote/service/__init__.py", line 198, in _load
File "PyLib/kernel/remote/service/__init__.py", line 149, in _initializeComputeNodes
File "PyLib/kernel/remote/service/__init__.py", line 78, in _makeMultiportArgs
File "PyLib/kernel/remote/service/__init__.py", line 41, in getParallelOptions
File "PyLib/cosimulation/partitioning/machinelist.py", line 65, in loadMachines
File "PyLib/cosimulation/partitioning/machinelist.py", line 258, in _constructMachineListSLURM
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
Traceback (most recent call last):
File "/uufs/chpc.utah.edu/sys/installdir/ansys/20.1/v201/SystemCoupling/PyLib/main/Controller.py", line 162, in
_run(sys.argv)
File "/uufs/chpc.utah.edu/sys/installdir/ansys/20.1/v201/SystemCoupling/PyLib/main/Controller.py", line 158, in _run
_executeScript(options)
File "/uufs/chpc.utah.edu/sys/installdir/ansys/20.1/v201/SystemCoupling/PyLib/main/Controller.py", line 98, in _executeScript
kernel.commands.readScriptFile(scriptFile)
File "PyLib/kernel/commands/__init__.py", line 31, in readScriptFile
File "PyLib/kernel/commands/CommandManager.py", line 169, in readScriptFile
File "/uufs/chpc.utah.edu/sys/installdir/ansys/20.1/v201/commonfiles/CPython/3_7/linx64/Release/python/lib/python3.7/site-packages/future-0.16.0-py3.7.egg/past/builtins/misc.py", line 82, in execfile
exec_(code, myglobals, mylocals)
File "run.py", line 37, in
Solve File "PyLib/kernel/commands/CommandDefinition.py", line 74, in func
File "PyLib/kernel/commands/__init__.py", line 28, in executeCommand
File "PyLib/kernel/commands/CommandManager.py", line 121, in executeCommand
File "PyLib/cosimulation/externalinterface/core/solver.py", line 137, in execute
File "PyLib/kernel/framework/ApplicationContext.py", line 30, in __getattr__
File "PyLib/kernel/framework/ApplicationContext.py", line 50, in __madeComponent
File "PyLib/kernel/framework/ApplicationContext.py", line 59, in
File "PyLib/kernel/framework/Annotate.py", line 50, in injected
File "PyLib/cosimulation/solver/__init__.py", line 65, in __init__
File "PyLib/kernel/fluxdb/FluxDB.py", line 336, in FluxDB
File "PyLib/kernel/remote/RemoteObjects.py", line 202, in __init__
File "PyLib/kernel/remote/RemoteObjects.py", line 88, in __init__
File "PyLib/ComputeNodeCommand.py", line 81, in newfunc
File "PyLib/kernel/remote/RemoteStatus.py", line 18, in forceRemoteStart
File "PyLib/kernel/framework/ApplicationContext.py", line 157, in __call__
File "PyLib/kernel/framework/ApplicationContext.py", line 152, in __call__
File "PyLib/kernel/framework/ApplicationContext.py", line 61, in _makeComponent
File "PyLib/kernel/framework/ApplicationContext.py", line 59, in
File "PyLib/kernel/framework/Annotate.py", line 50, in injected
File "PyLib/kernel/remote/service/__init__.py", line 244, in __init__
File "PyLib/kernel/remote/service/__init__.py", line 198, in _load
File "PyLib/kernel/remote/service/__init__.py", line 149, in _initializeComputeNodes
File "PyLib/kernel/remote/service/__init__.py", line 78, in _makeMultiportArgs
File "PyLib/kernel/remote/service/__init__.py", line 41, in getParallelOptions
File "PyLib/cosimulation/partitioning/machinelist.py", line 65, in loadMachines
File "PyLib/cosimulation/partitioning/machinelist.py", line 258, in _constructMachineListSLURM
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

[Stephen Orlando]

Hi Martin Are you able to upgrade to the latest release 2021R2 (or even 2021R1)? There are some known issues with how System Coupling handles some machine names in 2020R1 which I think you might be encountering. These issues are fixed for 2021R1.
Steve