On their side now, he's still trying to catch up.
Ellis was officially removed from the preseason physically unable to perform list, meaning he cannot be placed on the list when the regular season begins. He wouldn't delve into the specifics of his absence, only saying he needed to get into shape.
"I'm lifting weights, just running, extra conditioning, just trying to get caught up to speed with these guys," he said. "They've been running a lot, so I'm just trying to get caught up with them and get ready for the season."
"Whatever Belichick's wanting me to do," Ellis said, "that's what I'm going to do."
Playing in all but six regular-season games throughout his career in New York, Ellis totaled 72.5 sacks, his best season coming in 2003 when he had 12.5 sacks with 69 tackles, 47 of them solo.
Now on the other end of this fierce rivalry, Ellis isn't surprised by what he sees so far. After all, he was drafted by former Patriots coach Bill Parcells and played three seasons under former New England defensive coordinator Eric Mangini, also a former disciple of Belichick.
"I was born into this family with Parcells, being drafted by him and going through that period my rookie year, and going through it with Mangini, you had to get back into the swing of things," Ellis explained. "It's a lot different, definitely a lot different, but it's a good thing."
"Just their winning attitude and how they go about things," he said of why he chose New England. "I've been on the other side for so long, you kind of respect what they do. I respect the New England Patriots. I did respect them. Once (Belichick) told me he wanted me here, I was like, 'Yeah, I'll come.'"
The Patriots, though, haven't appeared to miss Ellis too much in their first two preseason games. New England limited the Jacksonville Jaguars to 193 yards of total offense in a 47-12 rout in the opener and delivered an even more impressive encore Thursday night against the Tampa Bay Buccaneers. Wreaking havoc all night, the Patriots held the Buccaneers to just 64 yards rushing and sacked Josh Freeman twice in the first quarter in a 31-14 win.
And to think, it all occurred without Ellis, in addition to mercurial defensive lineman Albert Haynesworth, an offseason acquisition from the Washington Redskins who returned to the practice field Saturday to watch his teammates after being absent for most of training camp.
Belichick said after practice that Haynesworth was "still day to day," according to the team's official site.
"I think the one thing that we have is a high level of character, just as far as on and off the field," Carter said. "I think when you have a great number of guys, it's not about ego, but it's guys coming together and willing to work and have a big sense of camaraderie. You can definitely go a long way. But like I told you, you got to earn it every play and every game."
Watching his teammates succeed, Ellis said, has only furthered his drive to become a part of it.
"Everybody knows when you start a defense, it starts up front. It's where it starts at. Then you go up to the linebackers, secondary," he said. "I think we got the potential to do great things with this defense."
The Associated Press contributed to this report.