Layton passed away at his home in Toronto early on Monday surrounded by his wife and children, his family said in a statement.
His left-leaning New Democratic Party (NDP) surged to become the official opposition for the first time in May's federal elections.
He stepped down only last month as party leader to fight his illness.
"We deeply regret to inform you that the Honourable Jack Layton, leader of the New Democratic Party of Canada, passed away at 04:45 local time [08:45 GMT] today," his family said in a statement.
"He passed away peacefully at his home surrounded by family and loved ones."
Prime Minister Stephen Harper saluted "Jack's contribution to public life, a contribution that will be sorely missed".
Layton announced he had been diagnosed with prostate cancer in February 2010.
He campaigned in spring this year, leaning on a cane after surgery for a broken hip, leading his party to a second-place finish in May's federal elections.
On 25 July, the former Toronto city councillor said he was quitting temporarily as NDP leader to fight a newly diagnosed "non-prostate cancer".
He was elected in 2003 as leader of the left-leaning New Democrats, long the third-place federal party in the Canadian parliament.
Before that, the Quebec native was an activist and community organiser in Toronto, campaigning on issues such as Aids, poverty, violence against women and homelessness.
He was elected MP in 2004 from a constituency in Toronto.
His popularity helped the NDP to overtake the Liberals in May's elections and become the country's official opposition party, when it took 103 seats, up from 37.
He is survived by his Hong Kong-born wife Olivia Chow, also an NDP MP, and his two children from a previous marriage, Michael and Sarah.
Funeral details have not yet been announced.