Councils in 'ghost town' warning