Trailing 5-2 to start the third period, the Rangers had a great chance to get back in the game with a two-man advantage for 85 seconds but failed to capitalize.
The Rangers came into the game with points in 17 of their last 19. They had chances throughout the night, but Forsberg was solid when called upon.
Trailing 2-0 after the first, New York looked to get back in the game and while they had a better second period in the end the Senators led 5-2 after 40 minutes.
Panarin had a power-play goal just 39 seconds into the second to make it 2-1, but Tarasenko regained the two-goal lead with his first in 13 games. Miller got the Rangers right back in it scoring 22 seconds later.
Tkachuk scored his second late in the period taking a rebound off the back boards and beat Shesterkin short side.
Tkachuk opened the scoring tipping an Artem Zub shot, and with just under two minutes remaining Giroux beat Shesterkin with a wraparound goal.
Ridly Greig returned to the Senators lineup after missing 10 games with a high ankle sprain.
Rangers: At Washington Capitals on Saturday.