Stern came to work for Greycroft after he retired from his role with the NBA.
"I don't want to portray it too strongly, I mean, we didn't pay him," Patricof joked. "I think he was just intrigued with the idea. I happened to meet him at the right time ... He hadn't figured out what he was going to do afterwards, and I said 'Why don't you come and join us on a regular basis every Monday morning, and we'll sit and talk'."
As a joke, Patricof suggested Stern order business cards that called him an "intern."
"Everyone at the firm thought I was crazy, but David loved to carry it around," Patricof said. "And, of course, he really was a special adviser to the firm, and it was very helpful, not just in sports, but in companies that required real managerial talent."
Stern stepped down from running the league in 2014 after a 30-year run overseeing explosive growth in the popularity of the game and league revenues.
Stern guided the basketball league into its most lucrative era: In 1983, the NBA’s revenues totaled $118 million. In 209, the league topped $8 billion.
Patricof said Stern also gave "insightful" and "responsive" viewpoints on things.
"The thing I liked best about him was if he didn't know something, he would ask the dumb questions which everybody else wanted to ask, but they didn't have the guts to do it," Patricof said.
Patricof said Stern became a venture capitalist, investing in companies alongside Greycroft.
"I couldn't contain his enthusiasm for some of these companies," Patricof recalled. "I'm interested to see what that portfolio is going to look like."
Stern passed nearly three weeks after suffering a brain hemorrhage. He was 77.
"He is going to be sorely missed by a lot of people," Patricof said.
FOX Business' Thomas Barrabi contributed to this report.