Understanding material loss in metamaterials can indeed be challenging. The passivity plot you mentioned, which shows values around 0.99-0.97, indicates that your metamaterial is mostly passive over the frequency range you are examining. High passivity values close to 1 suggest that there is very little loss, which might seem contradictory if you are expecting higher losses.
When you calculate the total loss using S-parameters, you are considering both the reflection (S11) and transmission (S21) to account for the energy that is not lost. The total loss you've calculated as a small fraction of 1 is the energy that is neither reflected nor transmitted, which includes both material and radiation losses.
The confusion arises when the passivity is much greater than the total loss. This could be due to several factors, including the possibility that the material losses are indeed very small within the frequency range you are looking at, or there might be an issue with how the passivity is being calculated or interpreted.
To further understand the material loss, you might need to look into the effective material parameters such as the effective permittivity and permeability, which can be extracted from the S-parameters. The effective refractive index and impedance can be calculated, which then allows for the retrieval of the effective permittivity and permeability. However, it's important to note that the extraction process can be challenging due to the multivalued nature of the complex inverse cosine function, and choosing the wrong branch can lead to incorrect results.