Bring water, cinnamon stick and cloves to boil in 2-quart saucepan. Reduce heat to low; cover and simmer 10 minutes.
Stir in milk and sugar. Return mixture to simmer. Add tea bags; cover and remove from heat. Allow to steep 2 minutes. Strain tea mixture before serving.
Serve immediately or refrigerate to serve cold.