Justin Hartley Looks Back at ‘Inappropriate’ First On-Camera Acting Job: ‘This Is the Last You’ll See Me on TV’

Justin Hartley would rather not keep a Tracker on some of his earliest acting roles. 

The TV star, 47, appeared on Thursday’s episode of The Late Show with Stephen Colbert and opened up about his first-ever on-camera job — which didn’t end up being quite what he expected. 

As soon as the talk show host, 59, asked the star of CBS’ new series Tracker whether he remembered his first on-screen role, Hartley smirked because he had a feeling that the show was going to pull up his true TV debut. “So I was going to tell a lie and then I was like, ‘They probably have it,’” he admitted. 

The former This Is Us actor then explained where his television career all began. “I remember a beer commercial that I did, and it was a commercial that they had me ad-libbing for hours,” he explained. “They were like, ‘Do it different. Ad-lib.’ There were no lines.”

Tracker's Justin Hartley Says He's Finally Found 'Peace' and Wants to Leave the 'Drama on Screen' (Exclusive)

The Late Show did in fact unearth the old clip, which finds a young Hartley approaching a woman sitting alone at a bar and sliding her a key to his hotel room, much to her annoyance. Although it appears as though the two exchange words, the only sound that can be heard is the tune of an upbeat jingle. 

“That’s so inappropriate!” he said after the commercial played. 

Hartley jokingly added, “This is the last you’ll see me on TV, apparently. I’m gone.”

Aside from the crude tone of the ad, he also opened up about why the job ended up being a bit of a disappointment. “I was really proud of all my ad-libs that day. And then when I watched the thing — I told my mom, I was like, ‘I made it. I’m on a commercial’  — they play that silly jingle over the whole thing.”

Justin Hartley Says 'Every Night's a Date Night' with Wife Sofia Pernas — Even Cozy Evenings at Home (Exclusive)

While speaking to Colbert, the Emmy-nominated actor also reflected on how he played the titular role of Frosty the Snowman in a school production growing up. Hartley explained that he was first thrilled about being the lead, but realized later on he felt like “the worst actor” because he barely moved or spoke the entire show. (“The snowsuit was a dunce cap, really,” he quipped.)

After looking back on his first on-screen performance, he said, “It’s like the second time I’ve had no lines — the lead role, no lines!”

“Frosty the Snowman all over," Colbert added.

Justin Hartley Keeps Cool While Cracking Mysteries in Tracker: 'Never Let Panic Take the Wheel' (Exclusive)

Hartley has more than a few lines in his latest project, playing bounty-hunter-turned-private-eye Colter Shaw in the new CBS thrillerTracker.

The SAG Award winner opened up to PEOPLE in a recent interview about what fans can expect. 

He described his latest role as “a guy who’s got it all figured out. He's efficient. He says what he means, means what he says, and he doesn't talk too much."

Hartley also shared the main difference between Shaw and his beloved character Kevin Pearson on This Is Us. “I enjoyed the This Is Us job so much, it just was so much character-sitting there, not a lot of exercise. The guy didn't move around a lot!” he joked. “So it's good to change it up. It's good to get a role where you're doing a lot of physical stuff.”

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Tracker premieres after Super Bowl LVIII this Sunday on CBS, and This Is Us can be streamed on Hulu and Netflix.