The game, at Yankee Stadium, was a television classic that captivated the country and sparked its love affair with the NFL. If it wasn't the greatest title game ever, it was surely the most significant.
For the gambler, though, it was just a win.
"I've had plenty of good losers, too," Banker said. "You're never going to get them all right."
Maybe not, though Banker once had a streak of 13 straight Super Bowl winners picking against the spread. Good thing, because he's never had a real job and winners pay the bills.
They have over the years, starting when he began booking bets in the family candy store in Union City, N.J., where his father would take the neighborhood action in between dispensing sweets. The son was a good enough basketball player to get scholarship offers, but he became even better at betting sports.
He'll be 85 this year, and he can still recall the big bets and bad beats of a career spent betting on the fortune of sports. The phone still rings in his sprawling home with people asking him "Lem, who do ya like?"
If he's not the greatest sports bettor - and many think he is - he's certainly the one of the few who have been around this long. He was especially good at picking winners of big fights, not surprising since he was Sonny Liston's best friend when the fearsome slugger ruled the heavyweight ranks.
Right now it's all about football. Always is at this time of year, something Banker was reminded of this week when his manicurist gave him her pick for the Super Bowl while filing his nails.
That's good news for this city's bookmakers, who could do record business in the matchup between the Giants and the New England Patriots. The wise guys will push their usual six-figure bets across the counter, but it's the massive flood of money from the squares betting $20 bills that will determine who is favored and by how much next week in Indianapolis.
So, Lem, who do ya like?
"I'm laying the points and taking New England," Banker said. "It's really very simple. To me, Tom Brady is the best quarterback I've ever seen - and I've seen a lot of them back from when I was a kid and Sid Luckman was playing."
They won't be run out of business, of course. Bookies always recover, because there are always more squares with money in their pockets who think they know more than the guys across the counter who have the point spread thing down to a science. Casinos have lost money only once on the Super Bowl in the last 10 years, taking a beating in 2008 when the Giants - who were 13-point underdogs - beat New England, 17-14.
"It's probably the best matchup there could be," said Jimmy Vaccaro, a longtime bookmaker who helps run sports books in several casinos for Lucky's Race & Sports Book. "The general public rules these events and they like these teams."
Banker won't be wagering that much himself. He never did make huge bets, preferring to make his money on volume instead.
And it's not like the old days when he had runners in different cities finding the best lines from bookmakers to lay his action on. Certainly not like the time he took a Minneapolis bookmaker for $30,000, only to be told he wasn't going to get paid. He got the money the next day, after asking a friend with, shall we say, connections, to look into the matter.
"There wasn't a bookmaker dead or alive that I didn't beat," Banker said. "I had runners everywhere, in New York, Miami, Chicago, all seeing different numbers. But now it's all the same numbers everywhere."
Computers and corporations have replaced pencils and candy store bookmakers. Online betting will dwarf anything even Las Vegas takes in on the game.
It's enough to make an aging gambler yearn for the days he once knew.
"It's very, very tough now," Banker said. "If I had to do it all again, I couldn't do it. I'd be driving a taxi."
Tim Dahlberg is a national sports columnist for The Associated Press. Write to him at tdahlberg(at)ap.org or follow at http://twitter.com/timdahlberg