Toronto mayoral hopefuls Olivia Chow and John Tory said they weren't surprised when their rival Doug Ford dropped out of a lunchtime debate hosted by the Empire Club on Friday.

The debate would have been Ford's second since joining the race for mayor in the wake of his brother's Rob's withdrawal due to illness. The Empire Club charged $800 a table.

In a statement issued late Friday morning, Ford said he made the decision to not attend after learning about the cost of the sold-out event. He said he is "a fan of a good debate," but is not interested in doing so in front of halls filled with the "privileged few."

He added that he told the Empire Club that he was going to make himself available “if they'd like to hold another event in the future that is open to the public at no cost."

Ford's absence was apparently not a surprise to both Chow and Tory. They pointed to his city hall attendance record during key votes as a councillor for Ward 2 Etobicoke North.

"We shouldn’t be too surprised he’s continuing that kind of performance," Chow told reporters prior to the start of the debate.

Tory, who has been the focus of some of Ford's recent attack about his wealth and privilege, also referred to the councillor's absence at council meetings.

"I just think he’s afraid. That’s why he's not here," Tory told reporters ahead of the debate. "He says the reason he’s not showing up is because there’s a pricetag on the debate. But there was a free one last night that I was with Mr. Chow. There’s a free one tonight that I'm going to be at with Ms. Chow."

Not an 'elitist club'

The Empire Club's president Andrea Wood defended the group's decision to charge debate attendees, saying that they are not an "elitist club."

"We are very inclusive. We are not-for-profit. We do charge for our lunches. We charge so that we can cover the cost of bringing these types of events to the public," Wood said.

In his first debate appearance Tuesday night, Ford accused campaign frontrunner John Tory of having everything handed to him on a silver platter, prompting laughter from some in the crowd.

Ford's attacks, however, have come with no mention of his own wealth accrued through his involvement in the business inherited from his father.

"I am campaigning to be the Mayor of all people, not the Mayor of the privileged few," Ford said in his statement Friday.

"For this reason I have made the decision not attend any debate that will cost residents money to participate in."

With a report from CTV Toronto's Natalie Johnson