Platform

Platform

Topics related to optiSLang, HPC, DesignXplorer, Cloud and more.

Error in FSI – runs fine in PC but not on HPC

    • ansysuser
      Subscriber

      Hello,


       


      I am trying to get a coupled system to run.  I found that the system runs to completion on my PC, but I get an error when I try to run it on HPC.

       *** ERROR ***                           CP =       1.126   TIME= 18:040
       The FSIN interface names of the mechanical model are different from the
       participant region names of the system coupling. 


       


      Progress = 0; Message = "System coupling run completed with errors.  Fluid Flow (Fluent) (Solution) reported: The FSIN interface names of the mechanical model are different from the participant region names of the system coupling."


       


       


       

    • ansysuser
      Subscriber

      Anyone have any insight on this?

    • ansysuser
      Subscriber

        +====================================================================+
       |                    Data Transfer Information (2)                   |
       +====================================================================+

       +


      +
       | Data Transfer: Data Transfer                                       |
       +
      +

       Source : Fluid Flow (Fluent)                                         
         Region                = fsi                                        
         Variable              = force                                      

       Target : Transient Structural                                        
         Region                = Fluid Solid Interface                      
         Variable              = Force                                      

       General Information :                                                
         Name                  = Data Transfer                              
         Execute Transfer At   = Start Of Iteration                         
         Convergence Option    = RMS Change In Data                         
           Target Value        = 0.01                                       
         Under Relax. Factor   = 1                                          
         Ramping               = None                                       

       +
      +
       | Data Transfer: Data Transfer 2                                     |
       +
      +

       Source : Transient Structural                                        
         Region                = Fluid Solid Interface                      
         Variable              = Incremental Displacement                   

       Target : Fluid Flow (Fluent)                                         
         Region                = fsi                                        
         Variable              = displacement                               

       General Information :                                                
         Name                  = Data Transfer 2                            
         Execute Transfer At   = Start Of Iteration                         
         Convergence Option    = RMS Change In Data                         
           Target Value        = 0.01                                       
         Under Relax. Factor   = 1                                          
         Ramping               = None     

    • ansysuser
      Subscriber

        +====================================================================+
       |                    Data Transfer Information (2)                   |
       +====================================================================+

       +


      +
       | Data Transfer: Data Transfer                                       |
       +
      +

       Source : Fluid Flow (Fluent)                                         
         Region                = fsi                                        
         Variable              = force                                      

       Target : Transient Structural                                        
         Region                = Fluid Solid Interface                      
         Variable              = Force                                      

       General Information :                                                
         Name                  = Data Transfer                              
         Execute Transfer At   = Start Of Iteration                         
         Convergence Option    = RMS Change In Data                         
           Target Value        = 0.01                                       
         Under Relax. Factor   = 1                                          
         Ramping               = None                                       

       +
      +
       | Data Transfer: Data Transfer 2                                     |
       +
      +

       Source : Transient Structural                                        
         Region                = Fluid Solid Interface                      
         Variable              = Incremental Displacement                   

       Target : Fluid Flow (Fluent)                                         
         Region                = fsi                                        
         Variable              = displacement                               

       General Information :                                                
         Name                  = Data Transfer 2                            
         Execute Transfer At   = Start Of Iteration                         
         Convergence Option    = RMS Change In Data                         
           Target Value        = 0.01                                       
         Under Relax. Factor   = 1                                          
         Ramping               = None     

    • ansysuser
      Subscriber

      Here is the system coupling window.  The solution ran just fine on my PC.


       


       


    • Konstantin
      Ansys Employee

      1) how are you transferring the project from PC to HPC, and 2) is your HPC linux-based?

    • ansysuser
      Subscriber

      Hello, 


      Since posting several weeks ago I have made some progress.  I will updated the situation below:


       


      I have Mechanical export a .dat file. 


      I have Fluent export .cas.gz and .dat.gz


      I have the Coupler export .sci

      Then I put them all on the same Linux directory and call them with the slurm commands:


      module load ansys
      echo "Made to system coupling"
      /apps/r/ansys/v193/aisol/.workbench -cmd ansys.services.systemcoupling.exe -inputFile ${COUPLING} &
      echo "Made it past system coupling"
      fluent 3ddp -g -t${NPF} -driver null -ssh -scport=${HOSTPORT[0]} -schost=${HOSTPORT[1]} -scname="${JOB[4]}" < ${JOURNAL} > output.FLUENT &
      ansys193 -b -scport=${HOSTPORT[0]} -schost=${HOSTPORT[1]} -scname="${JOB[2]}" -i ${STRUCTURALDATA} -o output.ANSYS -np ${NPA}


      And in the slurm output file I see this:


      Made to system coupling
      Made it past system coupling
      [0] MPI startup(): Single-threaded optimized library
      ... lots of lines of MPI startup...
      Cleanup script file is /pfs/tsfs1/gscratch/....
      forrtl: No such file or directory
      forrtl: No such file or directory
      forrtl: No such file or directory
      forrtl: No such file or directory
      forrtl: No such file or directory
      forrtl: No such file or directory
      forrtl: severe (28): CLOSE error, unit 24, file "Unknown"


      In output.FLUENT I see it loads the case and data, builds mesh, distributed mesh, parts, faces, etc, then gets to Done. Then the next thing I see in there is:

      > Deleting Client...done
      Shutting down Server...done
      No error handler available
      System Coupling Service encountered fatal error.#f


      In the output.ANSYS file I see:

       *** ERROR ***
       Another ANSYS job with the same job name (file) is already running in this
       directory or the file.lock file has not been deleted from an abnormally
       terminated ANSYS run.  To disable this check, set the ANSYS_LOCK environment
       variable to OFF.


      Now I know the file.lock is deleted between each try of this, so that is not the problem.  Also, this is the only job running when I try to run it.  No other ANSYS jobs at all on the whole cluster.


       


      ANSYS wrote out a whole bunch of other files, so let me know if you need to know what is in those as well.


       


      Thanks


       


       


       


       


       

    • Stephen Orlando
      Ansys Employee

      Hi there,


      It looks like you're using the old command line method to start this case. Please review the following tutorial that goes over how to run System Coupling from the command line in 2019R1.


      https://ansyshelp.ansys.com/account/secured?returnurl=/Views/Secured/corp/v193/sysc_tut/sysc_tut_oscplate_commandline.html


      You can put all the System Coupling commands into a text file, (run.py for example, but the name isn't important). Then in your slurm commands you would have ...systemcoupling -R run.py which would start all participants.


      Steve


       

    • ansysuser
      Subscriber

       Thank you, I will give that a try if I can and let you know how it goes.

    • ansysuser
      Subscriber

      Hello,


      I have followed the above as far as setting up the problem and getting the files onto HPC. Then I use this command in the batch file:


      /apps/r/ansys/v193/SystemCoupling/bin/systemcoupling -R run.py > outsyscoup.txt

      The only thing that is in the outsyscoup.txt is:


      Executing from: /apps/r/ansys/v193/SystemCoupling/bin/systemcoupling
      ... list of Point Releases and Patches....
      executing script 'run.py'


      +


      +
      | The most recent execution of the LoadParticipants command reset the        |
      | datamodel.  All content has been cleared.                                  |
      +
      +


      The contents of run.py are as follows (I was hoping to get a printout of the names as in the documentation so I could then do the next step, i.e., use GetRegionNamesForParticipant and then modify the timestep).  But these commands did not print out the ChildNames, as you can see from the output above.

      LoadParticipants(InputFiles = )
      DatamodelRoot().CouplingParticipant.GetChildNames()


       


      What has gone wrong?


       


      Thanks


       


       


       

    • Stephen Orlando
      Ansys Employee

      - Is the current directory of your linux terminal the same as the location of the .scp files?


      - Are the .scp files located in the same directory? You could also have them in separate directories with: LoadParticipants(InputFiles = )


      - If you open both .scp files in a text editor, is the  set correctly?


      You could also use the workflow with setup in Workbench and execution in the command line. The workflow is a bit easier. I meant to send this link last time. https://ansyshelp.ansys.com/account/secured?returnurl=/Views/Secured/corp/v193/sysc_tut/sysc_tut_oscplate_sci-commandline.html.


      One other note, ANSYS doesn't officially support slurm.


      Steve

    • ansysuser
      Subscriber

      Hello Steve,


       


      Thanks for your follow up.


      For more background: I am working remotely on HPC, so I don't exactly have a "terminal" that I can use real time.  I have to use a scheduler and the cluster uses SLURM through bash files.  All commands to run a job have to be in the bash file. I have been using ANSYS on SLURM for over 5 years on different clusters, but only with Fluent, and I have never had a problem until using the system coupling.  


      Now, all files are in the same directory, including the bash file that initiates the job, and the ANSYS files.  I made sure that the in the scp files is set to the local directory on the cluster, so this is not the problem.  Here is a screenshot of the directory, showing all files inside it.



       


      I can try the other method you suggest and let you know if that works.


       


      Thanks again.

    • ansysuser
      Subscriber

      Hello,


       


      I tried to make it through that tutorial you linked to above, using the oscillating plate even, but there is a part where there seems to be something missing in the tutorial instructions.  On this page (which is the very first link of the page you gave)


       


      https://ansyshelp.ansys.com/account/secured?returnurl=/Views/Secured/corp/v193/sysc_tut/sysc_tut_oscplate_sci-commandline_steps.html


       


      Look under 1. where it says, "When setting up project systems, prepare input files for each system, (Structural, Fluid, and System Coupling) before closing the associated application"



      I went through the instructions under Structural and Fluid through those links, but when I click on the System Coupling link the instructions say, "On the Project Schematic, ensure that the System Coupling system's Setup cell (C2) is in an Up-to-Date (  state."  My System Coupling has a question mark.  The tutorial has not covered how to set up System Coupling.  It skipped that step.

      Help!

    • ansysuser
      Subscriber

      Hello,


      I was able to get to the system coupler setup by gong to the other track of the tutorial (setting up and solving in Workbench, as opposed to setting up in Workbench and solving at command line).  So when I ran the case this way, I get:


       


      Starting Command Console...
      Type ^Z to exit
      >>> Exiting Command Console...
      /var/spool/slurmd/job2752553/slurm_script: line 12: syntax error near unexpected token `FilePath'


      I am not sure why the error about FilePath is there, because this is the exact command simply copied and pasted from the tutorial linked above.


      I appreciate any further input, as I am trying to evaluate the performance of HPC for coupled analysis for our group. We want to know if it is worthwhile to purchase several HPC licenses if the runs are significantly faster on HPC for these types of simulations than on our high end desktops.  I have been trying to get a simple coupled simulation to run on HPC for over a month now. Fluent runs just fine, and so I have been able to compare performance, but this coupled simulation seems to be much more trouble!


       


      Thanks


       


       

    • Konstantin
      Ansys Employee

      could be a result of copy-pasting  Are you operating with text files with the script and systems coupling commands on windows and then using them on linux, or are you creating those files directly on linux? I am asking because there is a difference between end of line character on DOS and Unix which may lead to an error when using Windows-created scripts on linux. The fix is to save text files in Unix format, which many text editors allow, or to use dos2unix command to convert the file on linux.


      If this doesn't help, we'll need to see your slurm and systems coupling scripts 


       

    • ansysuser
      Subscriber

      Hello kkoubat,


      I have gotten further.  I am facing a new error, which is detailed below.  Recall that I am following Steve's advice for the setup under the tutorial "Setup with Workbench and run from Command line" for the oscillating plate model.

      Here is how I call ANSYS in the bash file:


      /apps/r/ansys/v193/SystemCoupling/bin/systemcoupling < inputfile.in


      And below is the contents of the file inputfile.in.  Note that this is taken directly from the tutorial.  I am aware of the eol difference across OS, so I made sure to paste into linux and then backspace and return over each line.  I don't think that is the issue.

      ImportSystemCouplingInputFile(FilePath = 'ScInput.sci')
      execCon = DatamodelRoot().CouplingParticipant
      execCon.ExecutionControl.InitialInput = 'mapdl.dat'
      execCon.ExecutionControl.WorkingDirectory = 'Structural_Run'
      execCon.ExecutionControl.InitialInput = 'fluidFlow.cas'
      execCon.ExecutionControl.WorkingDirectory = 'Fluid_Run'
      execCon.ExecutionControl.PrintState()
      execCon.ExecutionControl.PrintState()
      Solve()


      Now below is the content of the output from SLURM.  Note that it appears to be seeing the input files for both Fluent and Mechanical, and that system coupler seems to be starting.

      Starting Command Console...
      Type ^Z to exit
                 
      >>> >>> >>> >>> >>> >>> >>> CouplingParticipant:Solution
        ExecutionControl
          WorkingDirectory = Structural_Run
          ParallelArguments =
          InitialInput = mapdl.dat
          Option = ProgramControlled
          AdditionalArguments =

      >>> CouplingParticipant:Solution 1
        ExecutionControl
          WorkingDirectory = Fluid_Run
          ParallelArguments =
          InitialInput = fluidFlow.cas
          Option = ProgramControlled
          AdditionalArguments =

      >>> Host spawning Node 0 on machine "t416.cluster" (unix).
      /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/bin/fluent -r18.0.0 3ddp -mpi=ibmmpi -p -node -t1 -nodehomedir=False -flux -wait -nosge -mport 10.98.4.168:10.98.4.1687343:0
      ls: cannot access /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/cortex/lnamd64/cortex.*.*.*: No such file or directory
      Starting /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/multiport/mpi/lnamd64/ibmmpi/bin/mpirun -e MPI_IBV_NO_FORK_SAFE=1 -e MPI_USE_MALLOPT_MMAP_MAX=0 -np 1 /pfs/tsfs1/apps/el7-x86_64/r/ansys/v193/commonfiles/CPython/2_7_15/linx64/Release/python/bin/python2 /apps/r/ansys/v193/SystemCoupling/PyLib/kernel/Engine/ComputeNode.py /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/lnamd64/3ddp_node/fluent_mpi.18.0.0 node -mpiw ibmmpi -pic shmem -mport 10.98.4.168:10.98.4.1687343:0 -nodehomedir=False -wait -nosge



      ID          Hostname      Core  O.S.      PID   Vendor                     


      e0          t416.cluster  1/32  Linux-64  3320  Intel(R) Xeon(R) E5-2683 v4
      controller  t416.cluster        Linux-64  3002  Intel(R) Xeon(R) E5-2683 v4

      MPI Option Selected: ibmmpi




      ==============================================================================
      +============================================================================+
      |                                                                            |
      |                         Summary of Coupling Setup                          |
      |                                                                            |
      +============================================================================+
      ==============================================================================
                 

      +============================================================================+
      |                              Coupling Control                              |
      +============================================================================+
      |                                                                            |
      | Analysis Type :                                                  Transient |
      | Duration Control                                                           |
      |    Option :                                                        EndTime |
      |    End Time :                                                     10.0 |
      | Step Control                                                               |
      |    Maximum Number Of Iterations Per Step :                               5 |
      |    Minimum Number Of Iterations Per Step :                               1 |
      |    Time Step Size :                                                0.1 |
      |                                                                            |
      +============================================================================+
      |                               Output Control                               |
      +============================================================================+
      |                                                                            |
      | Output Control Option :                                       StepInterval |
      | Output Frequency :                                                       2 |
      |                                                                            |
      +============================================================================+
      |                        Participants Information (2)                        |
      +============================================================================+
      |                                                                            |
      | Participant: Structural                                                    |
      |    Type :                                                            MAPDL |
      |    Update Controls:                                                        |
      |       Option :                                           ProgramControlled |
      |    Region: Fluid Solid Interface                                           |
      |       Topology :                                                   Surface |
      |       Input Variables :                                              Force |
      |       Output Variables :                          Incremental Displacement |
      |                                                                            |
      | Participant: Fluid                                                         |
      |    Type :                                                           FLUENT |
      |    Update Controls:                                                        |
      |       Option :                                           ProgramControlled |
      |    Region: wall_deforming                                                  |
      |       Topology :                                                   Surface |
      |       Input Variables :                                       displacement |
      |       Output Variables :                                             force |
      |                                                                            |
      +============================================================================+
      |                         Interfaces Information (1)                         |
      +============================================================================+
      |                                                                            |
      | Interface: interface-1                                                     |
      |    Sidene                                                                |
      |       Coupling Participant :                                    Structural |
      |       Region List :                                  Fluid Solid Interface |
      |    Side:Two                                                                |
      |       Coupling Participant :                                         Fluid |
      |       Region List :                                         wall_deforming |
      |    Transfer To Side One:Force                                              |
      |       Side One Variable :                                            Force |
      |       Side Two Variable :                                            force |
      |       Convergence Target :                                        1.00E-02 |
      |       Ramping Option :                                                None |
      |       Relaxation Factor :                                         1.00E+00 |
      |       Mapping Type :                                  Surface Conservative |
      |    Transfer To Side Two:Incremental Displacement                           |
      |       Side One Variable :                         Incremental Displacement |
      |       Side Two Variable :                                     displacement |
      |       Convergence Target :                                        1.00E-02 |
      |       Ramping Option :                                                None |
      |       Relaxation Factor :                                         1.00E+00 |
      |       Mapping Type :                            Surface Profile Preserving |
      |                                                                            |
      +============================================================================+
      +============================================================================+
      |                     Participants Execution Information                     |
      +============================================================================+
      |                                                                            |
      | Structural                                                                 |
      |   Execution Command:                                                       |
      |     "/apps/r/ansys/v193/ansys/bin/ansys193" -b nolist -s noread -o Solutio |
      |     n.out -scport 42633 -schost t416.cluster -scname "Solution" -scid="332 |
      |     0_t416.cluster" -sclic=2325@tlic1 -i mapdl.dat                         |
      |   Working Directory:                                                       |
      |     REDACTED                     |
      |                                                                            |
      | Fluid                                                                      |
      |   Execution Command:                                                       |
      |     "/apps/r/ansys/v193/fluent/bin/fluent" 3ddp -g -scport=42633 -schost=t |
      |     416.cluster -scname="Solution 1" -scid="3320_t416.cluster" -sclic="232 |
      |     5@tlic1" -i Solution1.jou                                              |
      |   Working Directory:                                                       |
      |     REDACTED                            |
      +============================================================================+
      Awaiting connections from coupling participants...Fatal error in PMPI_Init_thread: Other MPI error, error stack:
      MPIR_Init_thread(805)....: fail failed
      MPID_Init(1832)..........: channel initialization failed
      MPIDI_CH3_Init(126)......: fail failed
      MPID_nem_init_ckpt(1087).: fail failed
      MPIDI_PG_SetConnInfo(711): PMI_KVS_Put returned 14
      In: PMI_Abort(69253391, Fatal error in PMPI_Init_thread: Other MPI error, error stack:
      MPIR_Init_thread(805)....: fail failed
      MPID_Init(1832)..........: channel initialization failed
      MPIDI_CH3_Init(126)......: fail failed
      MPID_nem_init_ckpt(1087).: fail failed
      MPIDI_PG_SetConnInfo(711): PMI_KVS_Put returned 14)
      srun: Job step aborted: Waiting up to 32 seconds for job step to finish.

    • Konstantin
      Ansys Employee

      hmm, looks to be mechanical end throwing MPI errors. This is ibmmpi. Let me bring our installation folks into this discussion

    • ansysuser
      Subscriber

      Thank you so much, kkourbat.

    • Stephen Orlando
      Ansys Employee

      Hi,


      As a sanity check, can you try: systemcoupling -R inputfile.in, instead of: /apps/r/ansys/v193/SystemCoupling/bin/systemcoupling < inputfile.in. Please post the output for this. I have a few other things to check if this doesn't work.


      Steve

    • ansysuser
      Subscriber

      Hello Steve,


      It looks the same to me.  I don't know if this helps or not, but I noticed when I look in my job queue that the node is still running this job even after no more output is printed to the output file.  It runs until the time requested is up.

      executing script 'inputfile.in'
      CouplingParticipant:Solution
        ExecutionControl
          WorkingDirectory = Structural_Run
          ParallelArguments =
          InitialInput = mapdl.dat
          Option = ProgramControlled
          AdditionalArguments =

      CouplingParticipant:Solution 1
        ExecutionControl
          WorkingDirectory = Fluid_Run
          ParallelArguments =
          InitialInput = fluidFlow.cas
          Option = ProgramControlled
          AdditionalArguments =

      Host spawning Node 0 on machine "t416.cluster" (unix).
      /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/bin/fluent -r18.0.0 3ddp -mpi=ibmmpi -p -node -t1 -nodehomedir=False -flux -wait -nosge -mport 10.98.4.168:10.98.4.168:41642:0
      ls: cannot access /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/cortex/lnamd64/cortex.*.*.*: No such file or directory
      Starting /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/multiport/mpi/lnamd64/ibmmpi/bin/mpirun -e MPI_IBV_NO_FORK_SAFE=1 -e MPI_USE_MALLOPT_MMAP_MAX=0 -np 1 /pfs/tsfs1/apps/el7-x86_64/r/ansys/v193/commonfiles/CPython/2_7_15/linx64/Release/python/bin/python2 /apps/r/ansys/v193/SystemCoupling/PyLib/kernel/Engine/ComputeNode.py /apps/r/ansys/v193/SystemCoupling/runTime/linx64/runTimeLibraries/launcher/fluent18.0.0/../fluent18.0.0/lnamd64/3ddp_node/fluent_mpi.18.0.0 node -mpiw ibmmpi -pic shmem -mport 10.98.4.168:10.98.4.168:41642:0 -nodehomedir=False -wait -nosge



      ID          Hostname      Core  O.S.      PID   Vendor                     


      e0          t416.cluster  1/32  Linux-64  5532  Intel(R) Xeon(R) E5-2683 v4
      controller  t416.cluster        Linux-64  5206  Intel(R) Xeon(R) E5-2683 v4

      MPI Option Selected: ibmmpi



      Cleanup script file is /pfs/tsfs1/gscratch/mfig/SysC_Run/cleanup-afd-t416.cluster-5206.sh


      ==============================================================================
      +============================================================================+
      |                                                                            |
      |                         Summary of Coupling Setup                          |
      |                                                                            |
      +============================================================================+
      ==============================================================================

      +============================================================================+
      |                              Coupling Control                              |
      +============================================================================+
      |                                                                            |
      | Analysis Type :                                                  Transient |
      | Duration Control                                                           |
      |    Option :                                                        EndTime |
      |    End Time :                                                     10.0 |
      | Step Control                                                               |
      |    Maximum Number Of Iterations Per Step :                               5 |
      |    Minimum Number Of Iterations Per Step :                               1 |
      |    Time Step Size :                                                0.1 |
      |                                                                            |
      +============================================================================+
      |                               Output Control                               |
      +============================================================================+
      |                                                                            |
      | Output Control Option :                                       StepInterval |
      | Output Frequency :                                                       2 |
      |                                                                            |
      +============================================================================+
      |                        Participants Information (2)                        |
      +============================================================================+
      |                                                                            |
      | Participant: Structural                                                    |
      |    Type :                                                            MAPDL |
      |    Update Controls:                                                        |
      |       Option :                                           ProgramControlled |
      |    Region: Fluid Solid Interface                                           |
      |       Topology :                                                   Surface |
      |       Input Variables :                                              Force |
      |       Output Variables :                          Incremental Displacement |
      |                                                                            |
      | Participant: Fluid                                                         |
      |    Type :                                                           FLUENT |
      |    Update Controls:                                                        |
      |       Option :                                           ProgramControlled |
      |    Region: wall_deforming                                                  |
      |       Topology :                                                   Surface |
      |       Input Variables :                                       displacement |
      |       Output Variables :                                             force |
      |                                                                            |
      +============================================================================+
      |                         Interfaces Information (1)                         |
      +============================================================================+
      |                                                                            |
      | Interface: interface-1                                                     |
      |    Sidene                                                                |
      |       Coupling Participant :                                    Structural |
      |       Region List :                                  Fluid Solid Interface |
      |    Side:Two                                                                |
      |       Coupling Participant :                                         Fluid |
      |       Region List :                                         wall_deforming |
      |    Transfer To Side One:Force                                              |
      |       Side One Variable :                                            Force |
      |       Side Two Variable :                                            force |
      |       Convergence Target :                                        1.00E-02 |
      |       Ramping Option :                                                None |
      |       Relaxation Factor :                                         1.00E+00 |
      |       Mapping Type :                                  Surface Conservative |
      |    Transfer To Side Two:Incremental Displacement                           |
      |       Side One Variable :                         Incremental Displacement |
      |       Side Two Variable :                                     displacement |
      |       Convergence Target :                                        1.00E-02 |
      |       Ramping Option :                                                None |
      |       Relaxation Factor :                                         1.00E+00 |
      |       Mapping Type :                            Surface Profile Preserving |
      |                                                                            |
      +============================================================================+
      +============================================================================+
      |                     Participants Execution Information                     |
      +============================================================================+
      |                                                                            |
      | Structural                                                                 |
      |   Execution Command:                                                       |
      |     "/apps/r/ansys/v193/ansys/bin/ansys193" -b nolist -s noread -o Solutio |
      |     n.out -scport 42400 -schost t416.cluster -scname "Solution" -scid="553 |
      |     2_t416.cluster" -sclic=2325@tlic1 -i mapdl.dat                         |
      |   Working Directory:                                                       |
      |                           |
      |                                                                            |
      | Fluid                                                                      |
      |   Execution Command:                                                       |
      |     "/apps/r/ansys/v193/fluent/bin/fluent" 3ddp -g -scport=42400 -schost=t |
      |     416.cluster -scname="Solution 1" -scid="5532_t416.cluster" -sclic="232 |
      |     5@tlic1" -i Solution1.jou                                              |
      |   Working Directory:                                                       |
      |                                 |
      +============================================================================+
      Awaiting connections from coupling participants...Fatal error in PMPI_Init_thread: Other MPI error, error stack:
      MPIR_Init_thread(805)....: fail failed
      MPID_Init(1832)..........: channel initialization failed
      MPIDI_CH3_Init(126)......: fail failed
      MPID_nem_init_ckpt(1087).: fail failed
      MPIDI_PG_SetConnInfo(711): PMI_KVS_Put returned 14
      In: PMI_Abort(69253391, Fatal error in PMPI_Init_thread: Other MPI error, error stack:
      MPIR_Init_thread(805)....: fail failed
      MPID_Init(1832)..........: channel initialization failed
      MPIDI_CH3_Init(126)......: fail failed
      MPID_nem_init_ckpt(1087).: fail failed
      MPIDI_PG_SetConnInfo(711): PMI_KVS_Put returned 14)
      srun: Job step aborted: Waiting up to 32 seconds for job step to finish.
      slurmstepd: error: *** STEP 2753601.0 ON t416 CANCELLED AT 2019-05-29T13:22:20 ***
      [mpiexec@t416.cluster] control_cb (../../pm/pmiserv/pmiserv_cb.c:798): connection to proxy 0 at host t416 failed
      [mpiexec@t416.cluster] HYDT_dmxu_poll_wait_for_event (../../tools/demux/demux_poll.c:76): callback returned error status
      [mpiexec@t416.cluster] HYD_pmci_wait_for_completion (../../pm/pmiserv/pmiserv_pmci.c:501): error waiting for event
      [mpiexec@t416.cluster] main (../../ui/mpich/mpiexec.c:1147): process manager error waiting for completion

    • Stephen Orlando
      Ansys Employee

      Thanks. We'll take a look into this in more detail and get back to you with what we find.


      Steve

    • Stephen Orlando
      Ansys Employee

      Hi,


      We're thinking this might be an MPI issues with Mechanical. As a test we need to see what happens when Mechanical is forced to run with Shared Memory Parallel. Please add the following line to your inputfile.in, run again, and send the output.


      execCon.ExecutionControl.AdditionalArguments = '-smp'


      Thanks,


      Steve


       

    • ansysuser
      Subscriber

      Hello,


      Thank you for your reply.  Here is the output where it deviates from above:


       


      +============================================================================+
      |                        Participants Information (3)                        |
      +============================================================================+
      |                                                                            |
      | Participant: Coupling Participant                                          |
      |    Type :                                                          DEFAULT |
      |    Update Controls:                                                        |
      |       Option :                                           ProgramControlled |
      |                                                                            |
      | Participant: Fluid                                                         |
      |    Type :                                                           FLUENT |
      |    Update Controls:                                                        |
      |       Option :                                           ProgramControlled |
      |    Region: wall_deforming                                                  |
      |       Topology :                                                   Surface |
      |       Input Variables :                                       displacement |
      |       Output Variables :                                             force |
      |                                                                            |
      | Participant: Structural                                                    |
      |    Type :                                                            MAPDL |
      |    Update Controls:                                                        |
      |       Option :                                           ProgramControlled |
      |    Region: Fluid Solid Interface                                           |
      |       Topology :                                                   Surface |
      |       Input Variables :                                              Force |
      |       Output Variables :                          Incremental Displacement |
      |                                                                            |
      +============================================================================+
      |                         Interfaces Information (1)                         |
      +============================================================================+
      |                                                                            |
      | Interface: interface-1                                                     |
      |    Sidene                                                                |
      |       Coupling Participant :                                    Structural |
      |       Region List :                                  Fluid Solid Interface |
      |    Side:Two                                                                |
      |       Coupling Participant :                                         Fluid |
      |       Region List :                                         wall_deforming |
      |    Transfer To Side One:Force                                              |
      |       Side One Variable :                                            Force |
      |       Side Two Variable :                                            force |
      |       Convergence Target :                                        1.00E-02 |
      |       Ramping Option :                                                None |
      |       Relaxation Factor :                                         1.00E+00 |
      |       Mapping Type :                                  Surface Conservative |
      |    Transfer To Side Two:Incremental Displacement                           |
      |       Side One Variable :                         Incremental Displacement |
      |       Side Two Variable :                                     displacement |
      |       Convergence Target :                                        1.00E-02 |
      |       Ramping Option :                                                None |
      |       Relaxation Factor :                                         1.00E+00 |
      |       Mapping Type :                            Surface Profile Preserving |
      |                                                                            |
      +============================================================================+
      +============================================================================+
      |                     Participants Execution Information                     |
      +============================================================================+
      |                                                                            |
      | Coupling Participant                                                       |

      +


      +
      | argument of type 'NoneType' is not iterable                                |
      +
      +

      Traceback (most recent call last):
        File "/apps/r/ansys/v193/SystemCoupling/PyLib/Controller.py", line 139, in
          _run(sys.argv)
        File "/apps/r/ansys/v193/SystemCoupling/PyLib/Controller.py", line 135, in _run
          _executeScript(options)
        File "/apps/r/ansys/v193/SystemCoupling/PyLib/Controller.py", line 107, in _executeScript
          kernel.commands.readScriptFile(scriptFile)
        File "PyLib/kernel/commands/__init__.py", line 31, in readScriptFile
        File "PyLib/kernel/commands/CommandManager.py", line 168, in readScriptFile
        File "inputfile.in", line 10, in
          Solve()
        File "PyLib/kernel/commands/CommandDefinition.py", line 72, in func
        File "PyLib/kernel/commands/__init__.py", line 28, in executeCommand
        File "PyLib/kernel/commands/CommandManager.py", line 120, in executeCommand
        File "PyLib/cosimulation/externalinterface/cosim_commands/solver.py", line 94, in execute
        File "PyLib/cosimulation/solver/__init__.py", line 105, in solve
        File "PyLib/cosimulation/solver/__init__.py", line 796, in __startParticipantsControlled
      cosimulation.solver.CosimulationError.CosimulationError
      Shutting down compute-node processes


       


       

    • Stephen Orlando
      Ansys Employee

      My mistake, it should be execCon.ExecutionControl.AdditionalArguments = '-smp', not 'Solution 2'

    • ansysuser
      Subscriber

      O.k., now we might be getting somewhere.  The new error looks to me like the coupled run started, but it thinks there is no Fluid-Structure-Interface in Mechanical.  But I know there is because I ran this on my PC first to make sure it would work before exporting.  Here is the error and a picture of my model tree which shows the FSI like created in the example.

      ==============================================================================
      +============================================================================+
      |                                                                            |
      |                              Coupled Solution                              |
      |                                                                            |
      +============================================================================+
      ==============================================================================
      +============================================================================+
      | COUPLING STEP = 1                                SIMULATION TIME = 0.1 |
      +


      +
      |                                     |      Source            Target        |
      +
      +

      |                           COUPLING ITERATION = 1                           |
      | Participant Solution encountered a fatal error. Message: No FSIN           |
      | interfaces are defined for the mechanical model.                           |

      ==============================================================================
      Stack backtrace generated for process id 8199 on signal 11 :

      +
      +

      | Solution: No FSIN interfaces are defined for the mechanical model.         |
      +
      +

      1000000: fluent() [0x6ae149]
      1000000: /usr/lib64/libc.so.6(+0x35270) [0x2b02bb834270]
      1000000: /apps/r/ansys/v193/fluent/../tp/IntelCompiler/2017.6.256/linx64/lib/intel64/libintlc.so.5(__intel_sse2_strcat+0x95) [0x2b02b56498f5]
      1000000: fluent(FatalError+0x35) [0x7048d5]
      1000000: fluent() [0x6ff3f5]
      1000000: fluent(_RpcServeOneRequest+0x1a1) [0x89d9d1]
      1000000: fluent(ServeUntilCondition+0x16) [0x703646]
      1000000: fluent(CX_PauseAtSyncPoint+0x132) [0x7031e2]
      1000000: fluent() [0x6d0d05]
      1000000: fluent(eval+0x497) [0x71d677]
      1000000: fluent(eval+0xd1e) [0x71defe]
      1000000: fluent(eval+0xd1e) [0x71defe]
      1000000: fluent(eval+0xd1e) [0x71defe]
      1000000: fluent() [0x71e6f6]
      1000000: fluent(eval_errprotect+0x4e) [0x71ed7e]
      Please include this information with any bug report you file on this issue!
      ==============================================================================

      Traceback (most recent call last):
        File "/apps/r/ansys/v193/SystemCoupling/PyLib/Controller.py", line 139, in
          _run(sys.argv)
        File "/apps/r/ansys/v193/SystemCoupling/PyLib/Controller.py", line 135, in _run
          _executeScript(options)
        File "/apps/r/ansys/v193/SystemCoupling/PyLib/Controller.py", line 107, in _executeScript
          kernel.commands.readScriptFile(scriptFile)
        File "PyLib/kernel/commands/__init__.py", line 31, in readScriptFile
        File "PyLib/kernel/commands/CommandManager.py", line 168, in readScriptFile
        File "inputfile.in", line 10, in
          Solve()
        File "PyLib/kernel/commands/CommandDefinition.py", line 72, in func
        File "PyLib/kernel/commands/__init__.py", line 28, in executeCommand
        File "PyLib/kernel/commands/CommandManager.py", line 120, in executeCommand
        File "PyLib/cosimulation/externalinterface/cosim_commands/solver.py", line 94, in execute
        File "PyLib/cosimulation/solver/__init__.py", line 107, in solve
        File "PyLib/cosimulation/solver/__init__.py", line 254, in __stepControlled

      Error:   File "PyLib/cosimulation/solver/__init__.py", line 358, in __iterate
      Cortex received a fatal signal (SEGMENTATION VIOLATION).
      Error Object: ()
        File "PyLib/cosimulation/solver/__init__.py", line 1112, in __reachSynchronizationPointAllActive
        File "PyLib/cosimulation/solver/__init__.py", line 1098, in __reachSynchronizationPoint
        File "PyLib/cosimulation/solver/__init__.py", line 1119, in __yieldControlTo
        File "PyLib/cosimulation/solver/__init__.py", line 1002, in __serveRequests
        File "PyLib/cosimulation/solver/__init__.py", line 1070, in __handleRequest
        File "PyLib/cosimulation/solver/__init__.py", line 1012, in __transmitExceptionToAllParticipants
      cosimulation.solver.CosimulationError.CosimulationError
      Shutting down compute-node processes


    • Stephen Orlando
      Ansys Employee

      Thanks, we'll continue investigating with this new information.


      Steve

    • Stephen Orlando
      Ansys Employee

      Hi,


      This does look like a setup issue, as you mentioned. Can you zip the run directory and attach it so that I can have a look?


      Steve

    • ansysuser
      Subscriber

      Hello Steve,


      I found the problem.  This is something your documentation team might want to fix to avoid future user confusion.  The oscillating plate FSI tutorial has several categories for how to run it as listed below.  Note that many users will select one of these tracts depending on their needs and my not even look at the other tracks.



      When I follow the link you gave for, "Co-Simulation with Workbench Setup and Command-Line Execution" it says to write the the structural input using tools>Write Input Files, which results in only one file (mapdl.dat) for use in the simulation (on this page):

      https://ansyshelp.ansys.com/account/secured?returnurl=/Views/Secured/corp/v193/sysc_tut/sysc_tut_oscplate_sci-commandline_setup_prepare_mech.html


      That is what I did last week.  However, if I had chosen the other route, "Co-Simulation with Setup and Execution from the Command Line" instead, I would have discovered that I was supposed to also write another file, not mentioned in the Workbench Setup tutorial.  Here I am directed to right click on Transient in the model tree and "Write System Coupling Files" which is a needed file (mapdl.scp).  Once I put the mapdl.scp in the same HPC directory as the mapdl.dat file I had from last week, it seems to have run successfully.

      Thank you for your time, Steve.  I appreciate it!

    • tchateauvert
      Subscriber

      Hello,


      I am trying to do the same thing but I am a bit confused with the different steps discussed in this forum. Is it possible for the ansysuser to present his final slurm commands along with all the files used and the files necessary in the working directory on the HPC cluster?


      Thank you,

    • Sidharath
      Subscriber

      Hi 


      Did you find something. I want to do this also. i am a  student i have no access to above links

    • maryam
      Subscriber

      Hi,


      I am trying to do the same thing for a few month but i could not find a way yet. as other user announced I am a bit confused with the different steps discussed in this forum. Is it possible to present your final job file or slurm commands along with all the files used and the files necessary in the working directory on the HPC cluster?


      I would appreciate if you could do this.


       


      Maryam

    • maryam
      Subscriber

      Hi,


      I am trying to do the same thing for a few month but i could not find a way yet. as other user announced I am a bit confused with the different steps discussed in this forum. Is it possible to present your final job file or slurm commands along with all the files used and the files necessary in the working directory on the HPC cluster?


      I would appreciate if you could do this.


       


      Maryam

    • maryam
      Subscriber

      Hi Steve,


      I saw this post and hope you can help me to solve this error.


      I am going to run system coupling in HPC cluster. I followed the Ansys tutorial for oscillating plate model step by step and exported all of the files we need.The only problem i faced was when i was going to add Data transfer 1 and 2. I got this error and i do not what is this error for?


      AddDataTransfer(Interface = "Interface-1",


                          TargetSide = "One",


                          SideOneVariable = "force",


                          SideTwoVariable = "FORC"


                          )... ... ... ... 


      Traceback (most recent call last):


        File "CommandConsole", line 4, in


        File "PyLib/kernel/commands/CommandDefinition.py", line 74, in func


        File "PyLib/kernel/commands/__init__.py", line 28, in executeCommand


        File "PyLib/kernel/commands/CommandManager.py", line 121, in executeCommand


        File "PyLib/cosimulation/externalinterface/core/dataModelInterfaces.py", line 157, in execute


        File "PyLib/cosimulation/datamodelinterfaces/__init__.py", line 176, in addDataTransfer


      RuntimeError: Invalid variable provided for side one. Variable does not exist: force.


      As i now in fluent setup, we define deforming and system coupling faces under dynamic meshing part. 


      I will appreciate if you or anybody could help me to solve this error.


       


      Maryam

    • maryam
      Subscriber

      Hi Steve,


      I am trying to run system coupling in Hpc cluster. I followed step by step the ansys tutorial for oscillating plate model and exported the files we need. i did not face any problem until when i was going to add Data transfer 1 and 2 in system coupling. i got this error.


      AddDataTransfer(Interface = "Interface-1",


                          TargetSide = "Two",


                          SideOneVariable = "force",


                          SideTwoVariable = "FORC"


                          )... ... ... ... 


      Traceback (most recent call last):


        File "CommandConsole", line 4, in


        File "PyLib/kernel/commands/CommandDefinition.py", line 74, in func


        File "PyLib/kernel/commands/__init__.py", line 28, in executeCommand


        File "PyLib/kernel/commands/CommandManager.py", line 121, in executeCommand


        File "PyLib/cosimulation/externalinterface/core/dataModelInterfaces.py", line 157, in execute


        File "PyLib/cosimulation/datamodelinterfaces/__init__.py", line 176, in addDataTransfer


      RuntimeError: Invalid variable provided for side one. Variable does not exist: force.


       


      As i know we define deforming and system coupling faces in the fluent setup under dynamic meshing part and i did it. but about defining the force and displacement  variable we define them in system coupling not in fluent.


      i will appreciate if you or anybody else could help me to solve this problem.


      Maryam

    • maryam
      Subscriber

      Hi Steve,


      I am trying to run system coupling in Hpc cluster. I followed step by step the ansys tutorial for oscillating plate model and exported the files we need. i did not face any problem until when i was going to add Data transfer 1 and 2 in system coupling. i got this error.


      AddDataTransfer(Interface = "Interface-1",


                          TargetSide = "Two",


                          SideOneVariable = "force",


                          SideTwoVariable = "FORC"


                          )... ... ... ... 


      Traceback (most recent call last):


        File "CommandConsole", line 4, in


        File "PyLib/kernel/commands/CommandDefinition.py", line 74, in func


        File "PyLib/kernel/commands/__init__.py", line 28, in executeCommand


        File "PyLib/kernel/commands/CommandManager.py", line 121, in executeCommand


        File "PyLib/cosimulation/externalinterface/core/dataModelInterfaces.py", line 157, in execute


        File "PyLib/cosimulation/datamodelinterfaces/__init__.py", line 176, in addDataTransfer


      RuntimeError: Invalid variable provided for side one. Variable does not exist: force.


       


      As i know we define deforming and system coupling faces in the fluent setup under dynamic meshing part and i did it. but about defining the force and displacement  variable we define them in system coupling not in fluent.


      i will appreciate if you or anybody else could help me to solve this problem.


      Maryam

    • maryam
      Subscriber

      Hi Steve,


      I am trying to run system coupling in Hpc cluster. I followed step by step the ansys tutorial for oscillating plate model and exported the files we need. i did not face any problem until when i was going to add Data transfer 1 and 2 in system coupling. i got this error.


      AddDataTransfer(Interface = "Interface-1",


                          TargetSide = "Two",


                          SideOneVariable = "force",


                          SideTwoVariable = "FORC"


                          )... ... ... ... 


      Traceback (most recent call last):


        File "CommandConsole", line 4, in


        File "PyLib/kernel/commands/CommandDefinition.py", line 74, in func


        File "PyLib/kernel/commands/__init__.py", line 28, in executeCommand


        File "PyLib/kernel/commands/CommandManager.py", line 121, in executeCommand


        File "PyLib/cosimulation/externalinterface/core/dataModelInterfaces.py", line 157, in execute


        File "PyLib/cosimulation/datamodelinterfaces/__init__.py", line 176, in addDataTransfer


      RuntimeError: Invalid variable provided for side one. Variable does not exist: force.


       


      As i know we define deforming and system coupling faces in the fluent setup under dynamic meshing part and i did it. but about defining the force and displacement  variable we define them in system coupling not in fluent.


      i will appreciate if you or anybody else could help me to solve this problem.


      Maryam

    • shenouda
      Subscriber
      good dayn i try to run system coupling between fluent and transient mechanical but always i facing error please, any help will be appreciated,nn *** FATAL *** CP = 1.342 TIME= 21:27:03n Unable to connect to SC Service on through port 60300. n Insure that the service is running. nn *** FATAL ***n Random read error on unit= 16 possible corrupt or missing file.n I/O status error= 0nn
Viewing 36 reply threads
  • The topic ‘Error in FSI – runs fine in PC but not on HPC’ is closed to new replies.