Oliver Hudson Says Mom Goldie Hawn Has Been Cooking Him 'Great' Breakfasts Since He Moved Home

Oliver Hudson is reaping the benefits of living back at home with mom Goldie Hawn.

The Cleaning Lady actor stopped by The Late Show Thursday alongside his sister and Sibling Revelry podcast co-host Kate Hudson, and revealed that Hawn, 76, has been treating him to some culinary masterpieces as he stays at her and Kurt Russell's home while his own is being renovated.

"Mom cooks great breakfast in the morning," said Oliver, 45.

"Your mother cooks you breakfast in the morning?" replied host Stephen Colbert, impressed.

"Yeah, biscuits and gravy," Oliver replied.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human-interest stories.

Oliver told E!'s Daily Pop earlier this week that he, wife Erinn and their three kids — Rio Laura, 8, Bodhi Hawn, 11, and Wilder Brooks, 14 — have moved in with Hawn and Russell, 70, temporarily as their house undergoes a remodel.

And Kate, 42, has a bone to pick with her big brother about it. As she teased him Thursday on The Late Show, "The irony of this ... [is that] I live seven blocks from my mother. ... I bought the house we grew up in. Our other brother [Wyatt Russell] lives, like, seven blocks from me."

"And Oliver has all kinds of places to be staying at with the kids — like, it would make more sense for, actually, Oliver and his wife and the kids to stay with me because then the cousins could be together, but he chose Mommy's house," she quipped.

"Well, you're not making biscuits and gravy in the morning," Oliver joked, laughing.

RELATED VIDEO: Oliver Hudson Shows Off Mom Goldie Hawn and Kurt Russell's Aspen Home: "Curtains Are 30 Years Old"

It's been quite the family affair for the Hudson/Hawn/Russell clan, lately. Earlier this month, Oliver appeared on Live with Kelly and Ryan virtually from Hawn and Russell's Aspen, Colorado, home while the family vacationed together.

He said, "Mom built this house first and then my stepdad Kurt built the other house. So we lived here in 1985 for two years."

Oliver said Aspen is a special place for him and his family, who've been visiting the Colorado town for 37 years: "It's a bit of a time capsule. Those curtains that you see are like 30 years old, you know what I mean? Mom refuses to update, which kind of is great."

"I went to elementary school up here," he added. "I actually took my kids out of school five years ago in L.A. and took them up here to live in the mountains and have some independence and some freedom. It's a really special place for sure."