If penalty based contacts are used to tie the parts together, this could introduce artificial stiffness that can affect the free modes. In particular this could affect the three rotational modes. To avoid this try changing the contact formulation to MPC. This should result in six rigid body modes with zero frequency