Steve Valiquette made 32 saves and Markus Naslund scored his seventh goal of the season for the Rangers, who have lost nine of their last 11 in Ottawa.
The Senators took control of the game with two goals in a span of 82 seconds late in the second. Winchester beat Valiquette with just over two minutes remaining for his second career goal, and Alfredsson made it 3-0 with his fifth of the season at 19:15.
Wade Redden, returning to Ottawa for the first time since signing with the Rangers in the off-season, helped set up Naslund's wrister early in the final period, but Foligno tipped Jarkko Ruutu's shot past Valiquette to make it 4-1 at 10:48.
Redden was greeted with a video tribute at the six-minute mark of the first period, and the crowd of 19,619 at Scotiabank Place gave the defenceman a standing ovation after the retrospective was finished. Redden spent 11 years in Ottawa before joining New York.
Spezza gave the Senators a 1-0 lead with a power-play goal five minutes into the game. Dany Heatley broke through the Rangers' defence to set up Spezza with a great pass.
Notes: The Rangers failed to register their first shot until the 13-minute mark in the first.