Drinkable in Taipei without treatment but still best to boil or filter. Filtered hot and cold water from dispensers is available in every hotel, guesthouse and visitor information centre so it's handy to bring your own bottle. Ice is usually fine at restaurants. Shaved ice (with fruit) is usually fine but take a look at the conditions in the shop.
According to the CDC tap water is generally not safe for consumption.
According to the Numbeo water drinkability and accessibility index, tap water has a score of 55.32 / 100 (100 being best).
Numbeo considers this score to be "Moderate"
1 adult could save about 869$ per year in Taiwan by drinking tap water instead of buying bottled water.
๐ณ Drinking tap water would also help with reducing your plastic footprint and improve environmental pollution.