Solve time question

- September 27, 2022 at 8:19 pm
  
  jmoss
  Subscriber
  
  Does increasing the number of available cores on an RSM HPC cluster improve solve times?
- September 27, 2022 at 11:05 pm
  
  peteroznewman
  Subscriber
  
  It depends…
  Are you solving CFD or Structural models? Since you posted in the Mechanical section I will assume you have a Structural model.
  How many cores did you start with?
  On a Structural model, the solve time has diminishing improvements with added cores and can actually increase if you use too many cores. The CFD models scale much more linearly with increasing core count than the Structural models.
  How many cores are on one node? The communication speed between cores on the same node is an order of magnitude faster than the communication speed between nodes, so the best performance is when the job runs on a single node.
  Don’t use all the cores on the node, that will increase the solve time. Use at least one less than the maximum number of cores if the core count is small. If the core count is high, a smaller number of cores might take less time.
  Does the problem run in RAM (incore) or not? If the model doesn’t fit in RAM, it will be slowed down by that and adding cores might have little effect. If the model doesn’t fit in RAM, you should spend time reducing the size of the model until it will run in RAM.
- September 28, 2022 at 12:52 pm
  
  jmoss
  Subscriber
  
  Using 32 cores in Mechanical. Not sure how to verify what model sizes fit in RAM. Would element load balance ratio also be a factor to verify?
  Also, I’ve been told that we use Infiniband that allows the interconnect between nodes to communicate “very fast”
- September 28, 2022 at 9:48 pm
  
  peteroznewman
  Subscriber
  
  Oh yes! The load balance ratio is a big factor in structural model as to how useful adding cores will be.
  To figure out if the model fits in RAM when solving a Static Structural model, make sure to set the Solver to Direct. Then when the solution is complete, search in the Solution Output for the term incore and out-of-core. The text there will tell you if the solution was run incore (entirely in RAM) or out-of-core.
  If you are going to solve the same basic model many times, it is worth doing some benchmarking and see what the elapsed time is as you change the number of cores. Do 4, 8, 16 and 32 cores to get four points on a graph.
  Infiniband has many versions in its history, but yes, that is better than gigabit Ethernet.
- September 29, 2022 at 12:16 pm
  
  jmoss
  Subscriber
  
  thanks very much for the great information!