As for comments that Harbaugh wants the Ravens to lock up the fourth-year wideout to a new contract, Smith -- who is entering the final year of his rookie deal -- said he leaves that to the people who negotiate deals.
"I understand how the business side of it works," Smith said while talking about his upcoming charity basketball game at the University of Maryland on Saturday. "But I don't really worry about it. That's something that I feel like you let the people who are supposed to handle it, handle it.
"That's between my agent and the front office guys. I just try to be the best player that I can be for our team, and the other stuff takes care of itself."
The Maryland product had a breakout performance last season, catching 65 passes for 1,128 yards and four touchdowns.
Smith has been outspoken about the Ravens' offense, saying the lack of a threat across the middle stunted their passing game. He said he was left frustrated after wide receiver Anquan Boldin was traded to the San Francisco 49ers and tight end Dennis Pitta suffered a hip injury that knocked him out for the first 13 weeks of last season.
"When Dennis was hurt, that really messed up the play without Anquan. We really didn't have a man in the middle like that," Smith said. "I feel like Anquan's the best receiver in the league when it comes to working the middle of the field. We missed that with him leaving, but it was twice as bad when Pitta got hurt because that's something that he's really good at. We really kind of had a void inside there."
"Bringing Steve in, he can play outside," Torrey Smith said, "but I really think he's going to be able to fill that void in the middle."
"I think that it's definitely a loss in a way, but at the same time I understand where they're coming from with goal posts possibly leaning," he said. "It only happened like once or twice, but it can delay a game and someone can fall and get hurt.
"I understand it, but it doesn't mean I like it too much."