According to a statement issued by the ministry, 23 of the houses are owned by Croatian nationals, 18 by Hungarians, six by Bosnians and one by a Slovene national.
A large number of police were deployed in the area to prevent potential incidents.
The demolition started with the buildings owned by Hungarian investors, whose spokeswoman Krisztina Vass told reporters that the demolition drive could have "serious, even tragic consequences".
She said that the Hungarian investors had been notified by the Hungarian ambassador to Zagreb that Croatian Environment Minister Marina Matulovic Dropulic had promised him that finished houses would not be demolished.
According to Vass, the Hungarians built the houses in 2002 before the Croatian government adopted a regulation on the protected coastline and believe those buildings should be made legal.
Vass said that the Hungarians were being discriminated against because Croatian-owned houses were not being knocked down. She insisted that Hungarian investors had built the houses on a development site and that they had paid all the required dues.
"During construction, it was promised to us that everything would be sorted, but now our buildings are being pulled down," she said.
Vass declined to reveal at what price the Hungarian investors had bought the land and at what price they rented apartments in those buildings. She expressed hope that the demolition drive would be halted.