Not getting an adequate backup for Manning at some point over the next 13 years cost Polian and his son, Chris, their jobs.
Less than 24 hours after finishing a 2-14 season with Manning on the bench, team owner Jim Irsay started the housecleaning by firing the architect of the Colts' success and his hand-picked successor. Coach Jim Caldwell is safe, for now, though the evaluation continues.
"Shock, man, just shock," safety Antoine Bethea said after finding out the Polians were gone. "I knew there were going to be some things happening, but I didn't expect them to come so soon."
While swift moves are common right after the season for struggling NFL teams, stability has been a key part of the Polian regime.
Since making Manning the top overall pick in 1998, the Colts have celebrated 141 regular-season wins, 11 playoff appearances, six division titles, two AFC championships and one Super Bowl title. During that span, there have been only three head coaches, two offensive coordinators and one family making the personnel moves.
The problem in Indianapolis was losing, which came in bunches this season.
"I felt that it was time for a change, that there was a need for a change," Irsay said, flanked by his three daughters. "Bill had entered a role where he was less involved, but still quite a bit involved because of the lockout and Peyton's injury and the losing streak. He was around a lot more than he probably anticipated or I did. But it really was a question about both situations. I thought that it was time to change the personnel department on the football side of things that wasn't involved with the coaching."
Bill Polian declined to comment when The Associated Press contacted him by phone, but did issue a statement through the team.
"I'm grateful for all the support the fans have shown us in good times and bad," Polian said. "Indianapolis has been a wonderful place to live and work. Most of all, I would like to thank the players coaches and staff who have played the pivotal role in this magnificent journey. I will miss them all."
Those players became increasingly dismayed with Polian's comments, particularly those made on his weekly radio show. And the most vocal critic was Manning, probably the only person in the organization with more leverage than the team's vice chairman.
Manning missed the entire season with a neck injury after signing a five-year, $90 million deal in July, about six weeks before he needed his third neck surgery in 19 months.
When Polian told listeners to his show that he and Manning had discussed drafting his eventual successor and that Manning was "OK" with it, the four-time league MVP later said he and Polian had never discussed the 2012 draft and it would be inappropriate for him to have those discussions. And just before Christmas, Polian told reporters that Manning would fail his season-ending physical. Following Sunday's 19-13 loss at Jacksonville, Manning said: "That's news to me."
An even bigger problem was that the lack of a backup plan.
With Manning struggling to come back from May surgery on his neck, Indy brought 17-year veteran Kerry Collins out of retirement in late August with a $4 million contract. He didn't last a month.
Collins was replaced by Curtis Painter, who started well but struggled badly after Bill Polian said he felt "vindicated" by Painter's success.
The Colts lost their first 13 games, finished with the second-fewest victories since the team moved to Indianapolis in 1984 and wound up with the No. 1 draft pick.
Bill Polian repeatedly said he should be blamed if there was a talent deficiency.
Chris Polian, who was named the team's GM four years ago but didn't start making the day-to-day decisions until this season, got caught in the crossfire. He was seen leaving the team complex Monday afternoon, just about the time word leaked of the firings.
"To think that just less than less than two years ago we were getting ready to go to Miami and play in a Super Bowl, and we were cleaning off the confetti from the turf in Lucas Oil Stadium, it has been a very sharp decline," Irsay said. "Even after going 10-6 last year, if people had said you would have the No. 1 draft pick and go 2-14, nobody would have believed it."
Irsay has not established a timeline for finding a new general manager, though he said he would start contacting other GMs and teams Monday night.
No decision on Caldwell is expected to be made until after the GM is hired, though players have overwhelmingly supported him.
"One thing about Jim is that he's a stand-up guy, and guys respect that," cornerback Jerraud Powers said. "He's a great coach, a good guy, and it was unfortunate situation this year. But he never lost the locker room."
In fact, Irsay said, there's a good chance Caldwell will be back.
Whoever replaces Polian faces an abundance of decisions in a major rebuilding effort.
Manning never played after undergoing his third neck surgery in 19 months, a fusion of two vertebrae to fix a damaged nerve that was causing weakness in his throwing arm.
He did, however, start throwing to teammates in mid-December and received good reviews from both Bill Polian, who was watching, and running back Joseph Addai, who was catching balls. Doctors familiar with the procedure who did not treat Manning say he should return to a high level of play now that the fusion has healed.
If he is healthy, Irsay has promised to bring back Manning regardless of the $28 million bonus he is owed in March, potentially putting two franchise quarterbacks -- Manning and Andrew Luck -- on the same roster.
"I think I can co-exist with any player I've ever played with," Manning said Sunday. "I think I've always been a good teammate in that way."
If Manning returns, Irsay might be more likely to bring back former Pro Bowlers Robert Mathis, Jeff Saturday and Reggie Wayne, all of whom are now in their 30s, and keep other key high-priced cogs in the Colts' success such as Brackett and safety Melvin Bullitt.
But the Polians will no longer be making those decisions.
"Bill and I had a chance to have a second meeting and to have some tears and a hug, and that was good," Irsay said. "It's extremely difficult. Walking out of the locker room in Jacksonville that was a hard walk, and it's hard because my affection for them is deep."