General

General

POLYFLOW – How to evaluate the calculation time for a large grid?

    • FAQFAQ
      Participant

      A customer has a very large grid, say 6e5 elements. What can be said about the calculation time? What can be said about the solver to use? What about the decomposition? A suggestion If the user needs to run only one or two iterations, it is probably not to important to identify the best solver, and running with the default selection should be OK; If the user needs to run several calculations, it may be a good idea to identify the best option, and I would suggest running two iterations with the following options, and identify the best: – AMF, 1 proc – AMF, 2 procs – classic, 1 proc – classic, 2 procs – classic, 4 procs This makes sense if many runs are to be done. In addition, it is often more interesting to run simultaneously four jobs, each with 1 or 2 procs, than running one job with 4 or 8 procs., as the communication time grows, as the processors umber increases (assuming that the memory survives.) What about the best mesh decomposition in such a case? Considering the various possible topological combinations that may exist between elements, it is difficult to draw a general rule! For illustrating this, let us consider three grids having about 6e5 elements, but grid1 has 10x10x6000 elements, grid2 has 10x250x250 elements, and grid3 has 85x85x85 elements. – I would decompose grid1 into the number of procs (say 2), so that each proc would be used at its best/maximum capacity. Only at the end, when the resulting contributions are merged, I may loose a bit… – I would decompose grid2 into an even power of 2, say 4, 16, 64, (or a power of 4) and use two procs; I am not so sure that 4 procs would make a difference… – I would decompose grid3 into a power of 8, and use two procs; I am not so sure that 4 procs would make a difference… It is expected that the calculation time for the 6e5 elements be large, although something similar has already run. It may be useful to evaluate the calculation time as follows. We consider several cubic grids, with about equally spaced elements, and successively involving 1e3, 3e3, 1e4, 3e4, 1e5 (and perhaps 3e5) elements, These are selected as they are more or less equally spaced on a log scale. On these grids, one runs a simple heat conduction case, with classical solver and 2 procs, and with the AMF solver. One can already evaluate the calculation time needed, which is about of the order N^7, where N is the typical number of elements in a direction. Next, one does the same vor a velocity-pressure case, for the first three or four grids. Having tets may lead to differences vs. having hexs. When jumping from a single heat conduction cast onto a velocity-pressure calculation, the number of unknowns is multiplied by about 7 if the mini-element is used, and 4 if the linear velocity is used; Other information: on a NxNxN grid (only hexs), the calculation time is typically of the order N^7: for a single heat conduction case, each additional element on a large grid brings its own new unknown; there are thus about N^3 unknowns; the frontal width is N^2. This is less dramatic for tets.