Sorry just noticed your replies as the system stops sending reminder information.

If you do more details convergence testing, both RCWA and FDTD will reach the same result, in theory. In practice, there are different factors that affects the accuracy. What is more, the time and efforts you spend on it might not be proportional to the improvements of the results. As you showed, autoshutoff min 1e-7 and 1e-8 reaches the "same" result. did you use more finer mesh and thicker PML? in both cases are they terminated with the autoshutoff level or the simulation time? please get more in formaiton on FDTD converging testing:

When the mesh reaches certain level, other factors may dominate the errors, till to some point.

Both RCWA and FDTD solves the Maxwell's Equations however in different ways. There is nothing fundamentally block one solver to reach more accurate result, for whatever the Q values. As you may know, RCWA uses the Fourier modes analytically to represent the complicated fields. In principle the more K values the more accurate, provided that mesh and material (fitting) are proper. I do not find any limit for the maximal k-values except for the computer resources limit.

When you cannot see the difference, you may notice some when you use log plot.