Guilin Sun
Ansys Employee

Ripples can be due to different causes. In addition to PML layers, distances, and autoshutoff min as you mentioned, PML type also plays a role.  It seems this is a grating-like periodic device. If so please check the largest diffraction angle.If the differaction angle is very large, common PML settings are not sufficient to suppress reflection from PML, and at the extreme case where differaction angle is 90 deg, eveything will be reflected back 100% in theory. 

You can use visualizer to roughly check the diffraction by use of 10 by 10 periods. You can also use grating analysis to get the exact diffraction angles: Diffraction grating – Ansys Optics

Please note that sometimes much smaller autoshutoff min is required to remove the ripples.

 

Please try those two suggestions.