Those criminals have been active mainly in the western part of Kosovo, wearing black uniforms to make the situation more dramatic, Vittrup told a news conference in Pristina on Wednesday.
Vittrup said the police possessed information as to who those persons might be, and added that the local population gave little or no support to them.
He also suggested that the situation should not be exaggerated.
The UN Mission in Kosovo (UNMIK) has described the security situation as stable.
Also on Wednesday, the police in Pristina arrested 36 members of the "Self-Determination" movement on suspicion that they had written slogans against the UN Administration in Kosovo on UNMIK cars.
Three of them have been detained, the spokesman for the Kosovo Police, Refki Morina, said.
The movement opposes negotiations on Kosovo's political status and insists that the international community immediately recognise the independence of the province.