Learning Forum

[Daniel.dejan.gunde]

Hello,

I have a strange problem with a transient Fluent run. It just closes usually after 2 hours without any error and the error log file is empty. After opening the autosaved case file I can continue the calculation normally. What can be the reason behind it?

Best regards,

Daniel

[Rob]

At two hours or randomly after that point? What does the end of the .trn file look like?

[Daniel.dejan.gunde]

The .trn files look like this at the end:

Node 4: Process 17360: Received signal SIGSEGV.

==============================================================================

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   RANK 0 PID 15616 RUNNING AT DES-CZC42471Z8
=   EXIT STATUS: -1 (ffffffff)
===================================================================================

[Daniel.dejan.gunde]

It feels like it is "semi-random", it doesn't stop every time at the point but it is hard to get past a certain point during transient calculation. It may be of importance that I was using manual adaption of the mesh together with deactivation of certain zones. I do get warnings that zone deactivation is not supported with hanging nodes but the zones are succesfully deactivated after running mesh check and the iterating process runs otherwise normally and results seem to be correct.

[Rob]

Does something trigger (adaption or mesh activation) at the step the model fails? SIGSEV usually means Fluent was expecting data and didn't get it, and a Termination can indicate a node failure which could be IT related. What was going on in the few lines above what you posted? Ie you'll see iteration output, something, and then the failure message. 

[Daniel.dejan.gunde]

The adaptation is completely manual therefore nothing should trigger it. I'm also running the case on only one machine. The lines above the error message look normal (there is some additional output generated by an UDF):

 

turbulent viscosity limited to viscosity ratio of 1.000000e+05 in 35 cells 
 75278  1.5397e-04  3.5609e-10  1.8280e-10  2.5000e-12  1.0690e-07  4.6271e-08  0:00:52   60

 Reversed flow on 599 faces (79.6% area) of pressure-outlet 265.

 Reversed flow on 309 faces (29.3% area) of pressure-outlet 276.

 Reversed flow on 22 faces (100.0% area) of pressure-outlet 71.

 Reversed flow on 15 faces (100.0% area) of pressure-outlet 72.

 Reversed flow on 15 faces (100.0% area) of pressure-outlet 90.

 Reversed flow on 22 faces (100.0% area) of pressure-outlet 91.

 turbulent viscosity limited to viscosity ratio of 1.000000e+05 in 35 cells 

  iter  continuity  x-velocity  y-velocity      energy           k       omega     time/iter
 75279  7.9995e-05  1.7424e-10  8.8987e-11  1.1917e-12  9.7319e-08  4.1238e-08  0:00:52   59
!75279 solution is converged
Report definition evaluated at time-step has 1 values
Values correspond to time-step index:7114
*** omega value is: -0.056904
Report definition evaluated at time-step has 1 values
Values correspond to time-step index:7114
*** report turbine_inlet_x-velocity is: 0.000040
Flow time = 2.489900000000262s, time step = 7114
36662 more time steps
*** Load torque value is -0.000764
*** Load torque value is -0.000764

Updating solution at time levels N and N-1.
 done.

Updating mesh at time level N... 
==============================================================================

Node 4: Process 17360: Received signal SIGSEGV.

==============================================================================

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   RANK 0 PID 15616 RUNNING AT DES-CZC42471Z8
=   EXIT STATUS: -1 (ffffffff)

[Rob]

If you rerun from the last autosave does the error occur? The trigger is a time step update, so is there anything unique at that step?

[Daniel.dejan.gunde]

I was doing that in hope to get pass a problematic time step but there is no particular "strange" time step update. I also double checked the .trn files and it stops every time at different time step allthough always within 2 hours of every run. 

[Rob]

You've got plenty of RAM and disc space? Is it always on the time step update? 

[Daniel.dejan.gunde]

Yes, there is a lot of RAM left as the case is not that big. It's 2D and consisting of only about 850.000 elements. Yes, it seems that it happens always on the time step update. At each time step also the mesh is updated as dynamic mesh - layering is active. However, layering is not active on the zones where there are hanging nodes.

[Rob]

And the motion doesn't try and collapse any zones? How many compute nodes? 

[Daniel.dejan.gunde]

I'm not sure what do you mean here. The elements are colapsing and extruding normally during the layering process. Otherwise I get a clear error message. The zones where the motion is taking place are also not modified - they are exactly the same as in my previous succesful runs. But I deactivated some other zones around them and appended new ones. The appended nodes were also addapted several times using hanging nodes - manual adaption before starting calculation. I'm using 22 compute nodes, sometimes 18, in one machine.

[Rob]

Try on 5-10 nodes. Fluent loses efficiency somewhere between 50-100k cells per core depending on the physics being solved.  Additionally it's possible there are several partitions interacting with newly created/destroyed cells which cause the error. 

[Daniel.dejan.gunde]

It makes sense. I'll try that when we get our license server back to work.

[Daniel.dejan.gunde]

Incredible! I'm running now on 8 cores and it's been running for 5 hours now without an error. Thank you so much!!

[Rob]

You're welcome.