Workers were scheduled to begin removing Sanders' 35- by 60-foot picture from the exterior of Lucas Oil Stadium on Wednesday, according to The Indianapolis Star.
Colts chief operating officer Pete Ward told The Star that Sanders' image will be replaced by a giant horseshoe.
"There are so many players to consider," Ward said, "but you can't put everyone up there."
He added, "(The horseshoe) is a team symbol that we feel is appropriate."
Sanders signed a five-year, $37.5 million contract with the Colts before the 2008 season, but injuries limited him to just nine games over the next three seasons.
When Sanders was healthy, he was one of the league's top defenders.
Sanders played in 14 games in 2005, helping the Colts go 13-0 and earning the first of two Pro Bowl selections. A knee injury limited him to just four regular-season games in 2006, but when he returned for the playoffs, the Colts' run defense dramatically improved and was a key reason the team won Super Bowl XLI over the Chicago Bears.
Sanders was healthy again in 2007, playing in 15 games, earning another Pro Bowl trip and becoming the first Colts player to win the league's NFL Defensive Player of the Year award.
But even Sanders acknowledged that his reckless style of play likely would shorten his NFL career, and some fans even wanted the Colts to keep the safety on the bench until the playoffs started.
There was a reason for all of it.
Injuries limited Sanders to just six regular-season games in 2008. He missed the first five games in 2009 following knee surgery, then in his second game back, he tore the biceps in his left arm and missed the rest of the season.
Sanders didn't even make it through the first game last season. He tore his right biceps on the first defensive series of the season opener and didn't play again.
Sanders has 290 tackles, 3.5 sacks, two forced fumbles, three fumble recoveries and six interceptions as a Colt.
The Associated Press contributed to this report.