Taylor Swift had an extra special guest by her side during iHeartRadio’s Jingle Ball on Friday: her boyfriend Joe Alwyn.
The 27-year-old “Shake It Off” singer headed to the star-studded New York City event with Alwyn, with the couple spotted entering Madison Square Garden holding hands.
It’s a rare appearance for Swift and Alwyn, who secretly dated for months before going public with their relationship in May.
“Taylor and Joe are spending time together in N.Y.C. Taylor was in the best mood. She was smiling and waving to fans,” a source tells PEOPLE. “Joe stayed for the whole show and they later left together as well. They are staying at Taylor’s N.Y.C. apartment.”
Alwyn was also with Swift for her pre-Jingle Ball private party and the duo hung with the singer’s pals Blake Lively and Ryan Reynolds, Jack Antonoff and Camila Cabello backstage.
Later, during the show, Swift performed her hits “Shake It Off,” “…Ready for It?,” “I Don’t Want to Live Forever” and “Look What You Made Me Do.” She then brought out Ed Sheeran to perform their duet “End Game.”
An insider previously told PEOPLE that Swift’s family is hoping that her latest relationship will last.
“It’s a great relationship that everyone hopes will last. Joe could definitely be the one,” the insider said “Their relationship is very different from Taylor’s past ones. Just like Taylor, Joe is a homebody. He loves being an actor but doesn’t like the celebrity part of it. It’s pretty special to find someone you’re on the same page with.”
RELATED: How Taylor Swift & Joe Alwyn Plan To Make It Work When She’s On Tour
Along with the release of her latest album, Reputation, Swift announced a global arena tour — which will likely take her away from her new British boyfriend. However, a source previously told PEOPLE that the love birds are determined to make it work.
“When Taylor tours internationally next year, they will figure it out,” the source said. “This is not anything they are stressing about. Their relationship is amazing, because they are both committed to making it work. It’s just a very normal and respectful relationship, built on give-and-take. Joe is a gentleman and just a great guy to date.”