Damboa, 85km (53 miles) from the state capital Maiduguri, is now said to be deserted as many people have fled to neighbouring towns.
Boko Haram attacked the barracks in the town two weeks ago, killing several soldiers.
Nigeria’s defence ministry said the military had repelled that attack and killed at least 50 insurgents.
But residents say that since then the town has been defended by vigilante groups.
Policemen and soldiers have abandoned the people to their fate.
President Goodluck Jonathan on Tuesday asked the National Assembly for an extra $1bn to help the military fight Boko Haram.
But the lawmakers have now gone on a two-month recess without debating the request.
Nigeria’s soldiers, often outgunned by the terrorists, have frequently complained that the insurgents have superior firepower.