Rob Lowe has been famous for more than 40 years, but to hear him tell it, there's still nothing that compares to his '80s prime.

"It was crazy stuff," Lowe tells PEOPLE. "It's the kind of stuff you look back on and go, did that really happen?"

The actor, who recently turned 60, and has a lengthy resume of screen credits from his decades-long career that includes starring roles in movies like The Outsiders as well as TV shows like The West Wing, Parks and Recreation, and 9-1-1-: Lone Star. But in some ways, he's never felt more famous than in his Brat Pack days.

Columbia Pictures/Getty

In 1985's St. Elmo's Fire, Lowe starred alongside an ensemble of fellow young actors including Emilio Estevez, Andrew McCarthy, Demi Moore, Judd Nelson, Ally Sheedy, and Andie MacDowell, all playing recent graduates of Georgetown University struggling to start their adult lives. Lowe's character, Billy Hicks, was a saxophonist frat boy who became a particular audience favorite — something Lowe learned firsthand after the movie's release.

Want more movie news? Sign up for Entertainment Weekly's free newsletter to get the latest trailers, celebrity interviews, film reviews, and more.

St. Elmo's Fire was a summer movie, originally hitting theaters on June 28, 1985. A few months later, some fans decided to dress up as Billy Hicks for Halloween.

"It was surreal. It was the first time I ever saw someone dressed like me for Halloween," Lowe says. "I remember being on a balcony, looking down, and somebody getting out of a car dressed like my character with a saxophone over their back and the shaggy hair, earring, and the Hawaiian shirt."

Amy Sussman/WireImage

Lowe had been in several movies (starting with The Outsiders) by the time of St. Elmo's Fire, but the Joel Schumacher-directed film elevated him to a new level of celebrity.

"It was an incremental process to occupying that place in the culture that I did in the '80s, and it was a lot," said Lowe. "I'm super grateful that I can say that I had that in my life, because very few people get to be that person. Every decade there's a new crop and society demands it. It's fun to watch that unfold, having been there."

Given the cultural staying power of St. Elmo's Fire and how fondly some of those involved look back on it, a sequel doesn't seem out of the question. Earlier this year, Lowe confirmed that conversations were happening around a possible sequel, but that the development is still in "very, very, very, very, very early stages. So we'll see."

If it does happen, Lowe might have to prepare himself for seeing a whole new wave of Billy Hicks costumes.