According to the WHO, tap water is safe for consumption for 100% of the country.
Iceland has some of the cleanest water in the world and tap water is completely safe to drink. Locals find it amusing to see travellers buying bottled water when the same quality of water is available from the tap.
Geothermal hot water smells of sulphur, but the cold water doesn't smell.
According to the CDC tap water is generally safe for consumption.
According to the Numbeo water drinkability and accessibility index, tap water has a score of 96.13 / 100 (100 being best).
Numbeo considers this score to be "Very high"
1 adult could save about 1153$ per year in Iceland by drinking tap water instead of buying bottled water.
๐ณ Drinking tap water would also help with reducing your plastic footprint and improve environmental pollution.
Like in most countries, water quality and accessibility differs from location to location, therefore it's the best to check the information available on that place.