Greek Defence Minister Panos Kammenos told the BBC that illegal construction has contributed to one of the country’s worst-ever wildfire disasters.
He said building by residents between wooded areas was a “crime” that had resulted in blocking escape routes.
Mr Kammenos was confronted by angry locals as he visited areas devastated by fires east of Athens this week.
At least 81 people died and search teams fear they will find further victims as dozens more are missing.
Arriving in the fire zone in the town of Mati on Thursday, Mr Kammenos was surrounded by residents who accused him of abandoning them.
“You left us alone,” one person told the minister, who was accompanied by the local mayor and the head of the air force.
One man told the BBC he had reached the fire station to ask for help, and found firefighters unaware of the blaze.
Another resident complained that people had been advised to dive into the sea to escape, when most of the elderly were unable to reach the sea in the first place.
Many of the survivors were saved by the sea, but a large number of the victims in Mati were trapped close to a cliff-edge, unable to reach the water.
- From the scene: Families left homeless in Mati
- The photos that explain the world’s wildfires
- British man in hospital after Greek fires
Mr Kammenos, a combative figure who leads the nationalist Independent Greeks, rejected accusations that the government had failed to protect its citizens.
The actions of some residents had closed the roads to the beach, he argued.
“This is a crime from the past,” he said. “This coast of Athens, all these properties, the majority are without a licence, and they have occupied the coast without rules.”
Rafina Mayor Evangelos Bournous says one in four houses have been completely destroyed by the fire, while 50% more have been partially damaged.
A satellite map of the area revealed the scale of the fire damage, showing that more than half the 12,760 acres affected were residential areas.
Greek fires: Residents ‘worsened disaster’ by illegal building