The Colts said they will hold a news conference at 3 p.m. ET on Thursday to introduce Pagano. That announcement came shortly after a league source told NFL.com's Steve Wyche about the move.
Colts owner Jim Irsay recounted how Pagano accepted the offer.
It's the third time Irsay has turned to a defensive-minded coach since replacing his father as owner in 1997, first hiring Jim Mora in 1998 and then Tony Dungy in 2002. Jim Caldwell, who succeeded Dungy in 2009 but was fired earlier this month, was a quarterbacks coach.
"Great hire," Colts Pro Bowl defensive end Robert Mathis, who could become a free agent this offseason, told WXIN-TV. "His work speaks for itself -- what he did with Baltimore last year, he had those guys humming, so I think it's a good hire, and let's see where we go with it."
"Chuck is unorthodox," Suggs told The Associated Press. "He's like The Joker. You never really expect what he's going to do, and everything has a motive."
Said Ravens linebacker Paul Kruger: "Chuck has a leadership quality about him. He's humble, but he also knows when to take the reins and take charge. He doesn't try to dominate you in every meeting. He's just a coach that knows exactly how players are and what direction they need."
It will be the first head-coaching job for Pagano, 51, who just ended his first season as the Ravens' defensive coordinator after three years coaching their secondary. Baltimore finished the 2011 season as the NFL's third-ranked defense in both points allowed (16.6 per game) and total yards allowed (288.9 per game).
Pagano is the fourth Ravens defensive coordinator to land a head-coaching job in less than a decade. The others were Marvin Lewis (Cincinnati Bengals), Rex Ryan (New York Jets) and Mike Nolan (San Francisco 49ers).
Pagano began his coaching career in 1984 as a graduate assistant at USC and spent time at Boise State, UNLV, East Carolina and Miami before joining the Cleveland Browns in 2001 to coach their secondary. He was the Oakland Raiders' defensive backs coach in 2005 and 2006, then served as defensive coordinator at North Carolina before joining the Ravens.
Pagano's Colts staff won't include his brother, John, whom the San Diego Chargers promoted to defensive coordinator earlier this month. John Pagano, an assistant in San Diego for 16 seasons, told the team's official website that while he's happy for his brother, "I have the job I've always dreamed of."
The Associated Press contributed to this report.