Bring water to a boil in a large saucepan. Remove from heat; add tea bags. Cover and steep 5 minutes. Remove tea bags.
Stir in sugar and cocoa. Bring to a boil; reduce heat. Add milk, vanilla and spices. Cook and stir over medium heat until mixture is heated through, about 10 to 15 minutes. Do not boil. Strain, if desired. Pour into warm mugs. Garnish each with whipping cream and a sprinkle of nutmeg, if desired.