Williams, who was scheduled to meet Wednesday with the Broncos, is the third candidate to spurn John Elway, the new chief football executive in Denver. Last week, Atlanta Falcons offensive coordinator Mike Mularkey canceled his interview, and Jim Harbaugh left Stanford for the San Francisco 49ers without granting Elway an audience.
After discussing the opportunity with family, friends and Saints players, Williams believed he needed to remain in New Orleans with his current team after a tough playoff loss in Seattle, according to the source.
"He didn't feel like he could stand in front of the defense in a team meeting (Tuesday) morning and tell them the things they need to do to win another Super Bowl and then board a plane to Denver," the source said.
Williams, who will enter his third season as the Saints' defensive coordinator, would like to be a head coach again, sources said, under the right timing and conditions, but he isn't comfortable doing so now.
"Got some very thoughtful responses from you guys-Thanks!" Elway wrote on Twitter. "Enjoyed meeting with Rico and Dirk today. Both made great presentations."
Elway, who is leading the search, has said Denver's next coach should be as big a believer in Tim Tebow's promise as he is. Koetter, who golfs on the same Jacksonville course as Tebow does, said Tuesday that he sees a bright future for the hardworking but raw rookie left-hander who supplanted Kyle Orton as the Broncos' starting quarterback last month.
"He's a tremendous football player; Tim's definitely in the developmental phase," Koetter said, echoing Elway's remarks over the last week. "It's very difficult to step in from the type of offense he ran in college and move into an NFL-style offense. But if anybody can get it done, I'd never sell Tim Tebow short."
Unlike Koetter, Dennison didn't have to introduce himself to Broncos brass. He knew his way around the building and exchanged plenty of greetings with his former co-workers.
Because Dennison has spent 24 years in Denver, many consider him the front-runner.
Dennison is the only candidate who lacks the NFL head-coaching experience that Elway said was a must, but his deep organizational knowledge would seem to make up for that.
"I don't know if it's an advantage or disadvantage," Dennison said of his strong ties to the team. "I've been around the Broncos forever. ... How can I not be excited? I'm orange and blue all the way, but I work for Houston now."
Kubiak, who also has deep roots in Denver, said he'd miss Dennison but wouldn't mind losing him back to the Broncos.
"He deserves the opportunity," Kubiak said. "He's done a good job in this league, and I know it's a big day for him, so we wish him all the best; and if he gets it, we're happy for him, and if not, we're lucky he's back."
Dennison also interviewed for the Broncos' head-coaching job two years ago when McDaniels was hired.
"I've got an open mind coming in," Dennison said. "I've got some ideas, and either they like them or they don't. Whatever's best for the organization, they're going to do."
One of Dennison's ideas is an old one: returning to the zone-blocking, one-cut scheme that paved the way for the Broncos' dominant ground games of the late 1990s and early 2000s.
Denver's league-worst defense is a major point of emphasis with the search committee that includes Elway, general manager Brian Xanders and team president Joe Ellis.
"There's a lot of work to be had, and I'll leave it at that," Dennison said. "We'll have to get it fixed if that's the opportunity. Somebody has to do something."
The Broncos are set for pass-rushing star Elvis Dumervil's return next season. He missed all of 2010 with a torn chest muscle one year after leading the league with 17 sacks. But star cornerback Champ Bailey could leave as a prized free agent following Denver's decision to withdraw a contract extension offer in October.
Koetter wouldn't share his plan to fix the defense with reporters after he arrived at Denver International Airport, but he did say, "It's tough to be a (top) defense in this league when your best pass rusher is out all year. What a difference that guy makes."
Dennison was one of three assistants who stayed on staff following Shanahan's ouster in 2009. McDaniels was fired less than two years later amid the team's worst skid in four decades and a videotaping scandal.
He has spent his coaching career with three head coaches who are offense-minded, and fixing the Broncos' deficient defense, which ranked last in the league, is the team's top priority.
He has spent a dozen years as an offensive coach, and those offenses have ranked eighth on average. With Alex Gibbs retired, Dennison is the top teacher of the zone-blocking scheme. But he wasn't the primary play-caller in his four seasons as offensive coordinator.
And Dennison is an unknown commodity when it comes to how he'd serve as the face of a franchise. He maintained the much-derided code of silence during his years coaching the Broncos' offensive line, and the team is emphasizing transparency following McDaniels' alienation of the fan base.
The roster needs a major makeover after a franchise-worst 4-12 season, Denver's fifth in a row without a playoff berth.
The Associated Press contributed to this report.