Speaking to the Tennessean on Saturday, Adams admitted he was faced with the decision to keep Fisher or Young -- and both wasn't an option.
"When you have a problem, you have to fix it," Adams said. "The coach is a veteran guy, and Vince is a young guy, and Fisher is under contract, and he didn't want to have Vince playing for him.
"I was thinking we could get him and Vince together, but it wasn't going to work. It is one of those things where we had to buckle down and get it over with."
Young was a little bitter but thanked Adams and Fisher for giving him a chance. The 27-year-old quarterback also said he had nothing against Fisher, though he never felt like the coach trusted him or that he was his guy.
For Adams, who remained loyal to Young through his ups and downs with the team, the decision to either release or trade the quarterback was hard.
"Vince is a good man," Adams told the newspaper. "Vince realizes he probably stepped on a lot of toes and did some things he shouldn't have done. He is sorry it took place, but he understood what we decided to do, and I hope he's learned a lesson."
Young is 30-17 in his five NFL seasons, but he's just 13-14 against teams finishing a season at .500 or better. He also has battled questions over his work ethic, leadership and injuries. He suffered a season-ending thumb injury on his right hand Nov. 21, tossed his shoulder pads into the stands and told off Fisher in front of the locker room before storming out, beginning the end of Young's days in Tennessee.
"The way things were going, it was a decision that had to be made. It wasn't working out," Adams said. "I think Vince is going to be able to get himself back to normal and another team will be glad to have him. He knows things happened that shouldn't have happened, and he'll be more careful in the future."
Fisher, meanwhile, will serve out the final year of his contract, which will pay the coach $6.5 million, in a lame-duck situation. He is the NFL's longest-tenured coach with his current team, having just finished his 16th full season and coached his 273rd game with the Titans.
"With what I am paying him, I am not going to let him go and pay him a lot of money," Adams said.
Tennessee started this season with high expectations and was atop the AFC South at 5-2 before losing eight of its final nine games to finish 6-10, giving the team the eighth overall pick in the April draft. Changes to the coaching staff are expected to be part of Fisher's return.
Adams told the Tennessean he didn't consider giving Fisher an extension.
"If we don't start winning games, you wonder about the coaching," Adams said. "You look around the league right now, and a lot of teams are letting coaches go. I don't have to tell Jeff anything. He has been at this thing long enough to know what has to be done.
"I am 88 years old now, and I can't wait around until I am 98 to get to another Super Bowl game. Jeff knows that," Adams added. "I can walk on my own now. But at 98, I might not be able to do that. Who knows how long I'll be able to do that? So we have to start winning some games. I'm not getting any younger, you know."
The Associated Press contributed to this report.