Learning Forum

[komal.gada]

Hello,

I have a RHEL 6.9 HPC setup to run ANSYS 2020R1. I am submitting a job from my windows workstation using RSM through the workbench. I can run the job successfully when I am using Intel MPI in parallel run settings, but as soon as I run using default MPI, or if I run a design point update, I get the following error.

[[email protected]] match_arg (../../../../../src/pm/i_hydra/libhydra/arg/hydra_arg.c:91): unrecognized argument e (11/16/2021 8:39:54 PM) 

[[email protected]] Similar arguments:

[[email protected]] envall

[[email protected]] envlist

[[email protected]] errfile

[[email protected]] envexcl

[[email protected]] env

[[email protected]] HYD_arg_parse_array (../../../../../src/pm/i_hydra/libhydra/arg/hydra_arg.c:128): argument matching returned error

[[email protected]] mpiexec_get_parameters (../../../../../src/pm/i_hydra/mpiexec/mpiexec_params.c:1313): error parsing input array

[[email protected]] main (../../../../../src/pm/i_hydra/mpiexec/mpiexec.c:1771): error parsing parameters (11/16/2021 8:39:54 PM) 

Any help is appreciated.

Best,

Komal

[George Karnos]

Would need to play with settingKMP_AFFINITY
KMP_AFFINITY=disabled
KMP_AFFINITY=respect,explicit
KMP_AFFINITY=none
setting it to one of the 3 should work ..
Best Geo

[komal.gada]

Where can I find the KMP_Affinity?
How do I change it while running RSM job?

[George Karnos]

It is a system environment variable that can be set directly on the redhat machine.

[komal.gada]

I tried all the 3 recommended settings for KMP_Affinity:
1) KMP_Affinity=disabled
Running Solver : /ansys_inc/v201/fluent/bin/fluent --albion --run --launcher_setting_file "fluentLauncher.txt" --fluent_options "-gu -driver null -driver null-workbench-session -i \"SolutionPending.jou\"-mpi=ibmmpi -t32 -ssh" (11/19/2021 11:14:43 AM)
/ansys_inc/v201/fluent/fluent20.1.0/bin/fluent -r20.1.0 --albion --run --launcher_setting_file fluentLauncher.txt
*******************************************
WARNING:Found KMP_AFFINITY=disabled setting in the env
*******************************************
Warning:DISPLAY environment variable is not set.
Graphics and GUI will not operate correctly
without this being set properly.
Warning:DISPLAY environment variable is not set.
Graphics and GUI will not operate correctly
without this being set properly.
[mpiexec@n055.cluster.com] match_arg (../../../../../src/pm/i_hydra/libhydra/arg/hydra_arg.c:91): unrecognized argument e
[mpiexec@n055.cluster.com] Similar arguments:
[mpiexec@n055.cluster.com] envall
[mpiexec@n055.cluster.com] envlist
[mpiexec@n055.cluster.com] errfile
[mpiexec@n055.cluster.com] envexcl
[mpiexec@n055.cluster.com] env
[mpiexec@n055.cluster.com] HYD_arg_parse_array (../../../../../src/pm/i_hydra/libhydra/arg/hydra_arg.c:128): argument matching returned error
[mpiexec@n055.cluster.com] mpiexec_get_parameters (../../../../../src/pm/i_hydra/mpiexec/mpiexec_params.c:1313): error parsing input array
[mpiexec@n055.cluster.com] main (../../../../../src/pm/i_hydra/mpiexec/mpiexec.c:1771): error parsing parameters (11/19/2021 11:15:05 AM)

2) KMP_Affinity=none
Running Solver : /ansys_inc/v201/fluent/bin/fluent --albion --run --launcher_setting_file "fluentLauncher.txt" --fluent_options "-gu -driver null -driver null-workbench-session -i \"SolutionPending.jou\"-mpi=ibmmpi -t32 -ssh" (11/19/2021 11:16:51 AM)
/ansys_inc/v201/fluent/fluent20.1.0/bin/fluent -r20.1.0 --albion --run --launcher_setting_file fluentLauncher.txt
*******************************************
WARNING:Found KMP_AFFINITY=none setting in the env
*******************************************
Warning:DISPLAY environment variable is not set.
Graphics and GUI will not operate correctly
without this being set properly.
Warning:DISPLAY environment variable is not set.
Graphics and GUI will not operate correctly
without this being set properly.
[mpiexec@n055.cluster.com] match_arg (../../../../../src/pm/i_hydra/libhydra/arg/hydra_arg.c:91): unrecognized argument e
[mpiexec@n055.cluster.com] Similar arguments:
[mpiexec@n055.cluster.com] envall
[mpiexec@n055.cluster.com] envlist
[mpiexec@n055.cluster.com] errfile
[mpiexec@n055.cluster.com] envexcl
[mpiexec@n055.cluster.com] env
[mpiexec@n055.cluster.com] HYD_arg_parse_array (../../../../../src/pm/i_hydra/libhydra/arg/hydra_arg.c:128): argument matching returned error
[mpiexec@n055.cluster.com] mpiexec_get_parameters (../../../../../src/pm/i_hydra/mpiexec/mpiexec_params.c:1313): error parsing input array
[mpiexec@n055.cluster.com] main (../../../../../src/pm/i_hydra/mpiexec/mpiexec.c:1771): error parsing parameters (11/19/2021 11:17:26 AM)

Finally 3) KMP_Affinity=respect,explicit
Running Solver : /ansys_inc/v201/fluent/bin/fluent --albion --run --launcher_setting_file "fluentLauncher.txt" --fluent_options "-gu -driver null -driver null-workbench-session -i \"SolutionPending.jou\"-mpi=ibmmpi -t32 -ssh" (11/19/2021 11:19:08 AM)
/ansys_inc/v201/fluent/fluent20.1.0/bin/fluent -r20.1.0 --albion --run --launcher_setting_file fluentLauncher.txt
*******************************************
WARNING:Found KMP_AFFINITY=respect,explicit setting in the env
*******************************************
Warning:DISPLAY environment variable is not set.
Graphics and GUI will not operate correctly
without this being set properly.
Warning:DISPLAY environment variable is not set.
Graphics and GUI will not operate correctly
without this being set properly.
[mpiexec@n055.cluster.com] match_arg (../../../../../src/pm/i_hydra/libhydra/arg/hydra_arg.c:91): unrecognized argument e
[mpiexec@n055.cluster.com] Similar arguments:
[mpiexec@n055.cluster.com] envall
[mpiexec@n055.cluster.com] envlist
[mpiexec@n055.cluster.com] errfile
[mpiexec@n055.cluster.com] envexcl
[mpiexec@n055.cluster.com] env
[mpiexec@n055.cluster.com] HYD_arg_parse_array (../../../../../src/pm/i_hydra/libhydra/arg/hydra_arg.c:128): argument matching returned error
[mpiexec@n055.cluster.com] mpiexec_get_parameters (../../../../../src/pm/i_hydra/mpiexec/mpiexec_params.c:1313): error parsing input array
[mpiexec@n055.cluster.com] main (../../../../../src/pm/i_hydra/mpiexec/mpiexec.c:1771): error parsing parameters (11/19/2021 11:19:30 AM)

None of the settings work.

[Hunter Wang]

The error message of "/i_hydra/libhydra/arg/hydra_arg.c:91)" should for Intel MPI, but Fluent v201 was launched using -mpi=ibmmpi flag. Not sure what caused the issue.
If you could edit v201 installation, backup and edit /ansys_inc/v201/SEC/SolverExecutionController/sec/bootstrap/common.py, change from "initializeMpiTypeAsIbm" to "initializeMpiTypeAsIntel" in corresponding job schedule line, e.g. if your job scheduler is PBS, change:
('MpiType', [], [], { 'EnableAutoSelectMpi': 'True', 'BatchQueueType' : 'PBS' }, initializeMpiTypeAsIbm) to
('MpiType', [], [], { 'EnableAutoSelectMpi': 'True', 'BatchQueueType' : 'PBS' }, initializeMpiTypeAsIntel) Then default MPI for v201 Fluent RSM job should be changed to Intel MPI, both for Solution update through RSM and Design Point update through RSM.
Upgrade to v202 (2020 R2) if possible, default MPI type in 2020 R2 is changed from IBM MPI (in 2020 R1 and previous releases) to Intel MPI.

[komal.gada]

That solved the issue.
I cannnot update to 2020R2 as the HPC I am running on uses Torque with MOAB as the job scheduler and RSM 2020R2 doesn't have Torque with MOAB.
Thankyou for your help.