Fluids

Fluids

Topics related to Fluent, CFX, Turbogrid and more.

Problem during restarting of System Coupling

    • sampathsrikanth1999
      Subscriber
      Hello everyone. I am trying to run a 2-way FSI (Mechanical-Fluent) to model blood flow though an artery. I am running the simulation using 2023R2 version in our University's cluster. I encountered a problem while trying to restart the system coupling. The initial run ended successfully after 108ms. When I tried to restart from 90ms (previous restart point), it runs well for some time (upto 92 ms) and fluent throws an error which is as follows:
      200 Stored Partitions:
      ----------------------------------------------------------------------
      Collective Partition Statistics:       Minimum Maximum Total
      ----------------------------------------------------------------------
      Cell count                                      688             712      140092
      Mean cell count deviation            -1.8%          1.6%
      Partition boundary cell count         98              312       41847
      Partition boundary cell count ratio 13.9%        44.6%        29.9%
       
      Face count                                     1511          1620         287262
      Mean face count deviation             -3.2%          3.8%
      Partition boundary face count          109            392          24799
      Partition boundary face count ratio  7.1%          24.2%        8.6%
       
      Cell weights                                     7e+03          7e+03     1e+06
      Mean cell weight deviation                 -1.8%           1.6%
       
      Partition neighbor count                          3                   21
      ----------------------------------------------------------------------
      Partition Method Metis
      Stored Partition Count 200
      Done.
      *** Error in /sw/eb/sw/ANSYS/2023R2/v232/fluent/fluent23.2.0/lnamd64/3ddp_node/fluent_mpi.23.2.0': free(): corrupted unsorted chunks: 0x00000000176983e0 ***
      *** Error in/sw/eb/sw/ANSYS/2023R2/v232/fluent/fluent23.2.0/lnamd64/3ddp_node/fluent_mpi.23.2.0': free(): corrupted unsorted chunks: 0x00000000163d32b0 ***
       
      The mechanical part seems fine. These were the last lines of the output file:
       *** AVERAGE DISP INCREMENT ACROSS SOURCE INTERFACE. System Coupling Region (FSIN_1)
           AVERAGE SENDING DISPLACEMENT INCREMENT DX =  0.39397E-04
           AVERAGE SENDING DISPLACEMENT INCREMENT DY =  0.21293E-03
           AVERAGE SENDING DISPLACEMENT INCREMENT DZ =  0.23668E-03
       
       *** ERROR ***                           CP =     240.430   TIME= 23:33:12
       The system coupling service error notification: Error when making a
       remote call to participant Solution 1.
      "Mechanical_1/Solution_SC_008.out" 1344L, 70742C                
       
      I am very new to fluent and I am not familiar with parallel computing. Can someone help me on resolving this issue? Thanks in advance
    • Rob
      Forum Moderator

      The error looks to be linked to the partitioning. However with 140k cells I'd only run on two cores: with 200 it's very likely that partitions are clashing and causing the problem. Whilst Fluent does scale well, I'd typically want a minimum of 50-100k cells on a core; if you go significantly below that data traffic between cores may actually slow the solver down. 

    • sampathsrikanth1999
      Subscriber

      Thank you for your reply. I used to run it in my local machine in four cores. It takes so much time. I need data for 1 second with 1ms time step (I can't go above that. The simulations fails). So, I thought if I run it in more cores, it will run fast (I am a novice. Sorry if I am wrong). In my case in 12 hours with 200 cores, I got only 100 ms data. So, what should I do to run it efficiently? Thanks in advance

    • Rob
      Forum Moderator

      Time step size will dictate how many steps are needed. CPU speed/parallel may speed up each time step. Unfortunately you'll find there's a maximum benefit to increasing the core count.

      12 hours for 100 time steps is very slow: check to see which part of the simulation is taking the time: I know Mechanical doesn't scale anything like as well as Fluent. How many iterations does it take to converge each Fluent time step?

    • sampathsrikanth1999
      Subscriber

      Thanks for the reply. My case details are as follows:

      Fluent starts to converge after coupling iteration 8. It takes approximately 60 iteration to converge which goes down with the succeding coupling iterations

      I have a residual control as 1e-4. Fluent runs for 100 iterations max. The maximum number of coupling iteration is 30.

      My major problem is the fsi data transfer to structural. I have set an rms of 0.001 for data transfer. Even after 30 iterations the residual limit is not met.(In some time steps, it is achieved in as low as 12 coupling iterations).  The following is an example at last coupling iteration

                                 COUPLING ITERATION = 30                           |
      +-----------------------------------------------------------------------------+
      | Transient Structural                |                                       |
      |   Interface: interface-1            |                                       |
      |     Data Transfer                   |         Not yet converged             |
      |       RMS Change                    |     6.85E-04          6.54E-03        |
      |       Sum x                         |     1.22E-01          1.16E-01        |
      |       Sum y                         |     2.65E-01          2.59E-01        |
      |       Sum z                         |     4.09E-01          3.89E-01        |
      +-----------------------------------------------------------------------------+
      | Fluid Flow (Fluent)                 |                                       |
      |   Interface: interface-1            |                                       |
      |     Data Transfer 2                 |             Converged                 |
      |       RMS Change                    |     1.04E-04          1.20E-04        |
      |       Weighted Average x            |     4.14E-05          4.14E-05        |
      |       Weighted Average y            |     2.15E-04          2.14E-04        |
      |       Weighted Average z            |     2.35E-04          2.35E-04        |
      +-----------------------------------------------------------------------------+
      | Participant solution status         |                                       |
      |   Transient Structural              |             Converged                 |
      |   Fluid Flow (Fluent)               |             Converged                 |

      I think the data transfer is where the problem occurs. I use Quasi-newton stabilzation (0.01) (based on a similar thread in the forum regarding simulation of arteries) and I am ramping up the load to structural with URF 0.5. Without ramping, the structural elements gets highly distorted. Thanks in advance

       

    • Rob
      Forum Moderator

      Ideally Fluent will converge in 10-15 iterations, but given the geometry changes you may need a few more. Have you run a single Fluent model in steady state and passed the pressure field over to Mechanical? The set up sounds sensible, but if the initial fluid flow field tries to expand the artery wall too much you may need to review/rethink the settings or modelling approach. 

    • sampathsrikanth1999
      Subscriber

      Thanks for the reply Rob. The 2-way FSI was the problem given to me. So this is my first setup. My senior worked in a simple model instead of the actual model of the artery. Are you suggesting me to run the FSI in fluent itself instead of system coupling? Thanks in advance

    • Rob
      Forum Moderator

      Intrinsic FSI is a good option. I was more suggesting to check the pressure field from Fluent is giving a sensible deformation in Mechanical. It won't be the first model where the pressure from CFD results in a balloon in Mechanical. 

    • sampathsrikanth1999
      Subscriber

      Thanks for the reply Rob. I can understand what you are suggesting but unfortunately, I don't know how to do that. Should I check the total deformation in the structural with the data I have? 

    • Rob
      Forum Moderator

      Yes, check total deformation and rate of change. Note I know next to nothing about structural modelling so you may want to run Mechanical outside of the system coupling to see what the total defomation could be. 

    • sampathsrikanth1999
      Subscriber

      Thanks Rob for your help. It means a lot to me. I will try to do what you have suggested. One final question. I am providing ramping when transferring force to the structural participant. Does it mean, I have to ramp the diplacement transfer to fluent too? Because I am not providing any ramping when transferring displacement

    • Rob
      Forum Moderator

      I don't know, I think Fluent will remesh based on what's sent back, but if the change is too great it won't help accuracy. 

Viewing 11 reply threads
  • You must be logged in to reply to this topic.