The actor was best known for his role as councillor David Horton on the BBC show, which stared Dawn French in the title role.
Gary’s son Josh said he “peacefully passed away” on Monday morning.
In a statement, he said: “Classically trained, it was the theatre where he truly flourished and he leaves a legacy of entertainment that saw him frequent the boards of Broadway, the West End and our living rooms on the telly!
“He leaves behind his two grandsons, Cooper and Bayley, and his son Josh. We will all miss him terribly.”
Gary appeared on every episode of The Vicar Of Dibley, in all three seasons of the show and its numerous specials between 1994 and 2007.
He made his last appearance as David in a 2013 Comic Relief short.
Gary’s other TV credits included The Sweeney, Brush Strokes and Hotel Babylon, and he even featured in comedy sketch show French And Saunders.
He also provided voiceovers for adverts including for Green Flag, Kwik Fit, Branston and Marmite.
Gary was an accomplished Shakespearean actor and had performed with the Royal Shakespeare Company including in Much Ado About Nothing.
In a statement, BBC Comedy said of his death: “Gary was an incredibly talented actor who is fondly remembered by us for his numerous appearances on some of the nation’s most treasured BBC shows, and of course as councillor David Horton in The Vicar Of Dibley.
“Our thoughts are with his family and friends at this difficult time.”
Many other stars of The Vicar Of Dibley have died in recent years.
Owen Newitt actor Roger Lloyd-Pack died in 2014 and Liz Smith, who played Letitia Cropley, died in 2016.
Emma Chambers, who played Alice Tinker, died in 2018, as did John Bluthal who appeared as Frank Pickle, while Trevor Peacock, who played Jim Trott, died last year.