TORONTO -- Prime Minister Stephen Harper went home Saturday night, back in time to what he called the "small town in Toronto" of his childhood.

Speaking at a gala marking the 100th anniversary of Leaside, Harper reminisced about the east Toronto neighbourhood where he grew up in the 1960s.

He says back then, girls and boys were still segregated in the school yard, although some of the older boys were growing out their hair and the girls started wearing shorter skirts.

Harper says it was a place where the local pharmacist made house calls and you could knock on a neighbour's door if you needed help.

He says he has some political memories too -- namely the passionate national debate over a new Canadian flag in 1964.

Harper says emotions ran high and some neighbours even stopped speaking to each other.

So as a five-year-old boy, Harper says he plowed right into the debate and asked everyone on his street which design they favoured and why.

Some people, like his parents, favoured one of the two principal designs for a new flag, he said.

"The Harpers liked the one with the blue borders," Harper said with a smile, "and three maple leaves."

He remembers watching the Maple Leaf going up the flagpole outside his kindergarten class in 1965. "And very quickly, peace came back to the neighbourhood," Harper added.

Leasiders were united on one issue, he said. They fought to keep their independence as a municipality, a battle they lost in 1967 when they were amalgamated with the township of East York.

"In Leaside you were, and I was, part of a community in the most intimate and the very best sense of the term," he said.

The neighbourhood has changed since then and become more affluent, Harper said. His parents sold their home for $39,500 in 1971 -- "probably not the wisest financial transaction they ever made," he said. Homes cost more than 10 times that now.

But it's "still a pretty special place," he said.

"I have long told people who ask me -- because I do get asked this question in different parts of the country -- so where exactly do you come from," he said. "And I say to them, I grew up in a small town in Toronto."