The two were engaged in a disagreement in the third quarter before officials stepped in to separate the players, and no penalty flags were thrown.
Gilmore, who shadowed Brown for most of the contest, explained after the game what caused the confrontation.
"He did a little dirty move, grabbed my helmet, threw me down," Gilmore said, via Zack Cox of NESN. "That was super dirty. But other than that, just competing on the field."
Brown finished the game with 49 yards receiving and a touchdown on four catches, but the touchdown grab in the second quarter came with cornerback Jason McCourty in coverage.
"It was fun," Gilmore said. "He's one of the best receivers today. One thing that he's good at -- it's not his routes -- but his (timing) ... when he makes his move. He's very good at it."