Army spokesman Lt Col Didier Badjek said the soldiers had been disarmed and were now being accommodated in schools.
Clashes are said to be continuing in the border town of Gamboru Ngala.
Boko Haram on Sunday released a video in which it said it had established an Islamic state in the towns and villages it controls in north-eastern Nigeria.
The group’s five-year insurgency has intensified in recent months despite the deployment of thousands of extra troops to the worst-affected areas.
Last week, a group of soldiers refused to follow orders to go and fight Boko Haram, saying the militants were better equipped.
Insurgents also seized one of Nigeria’s two main police training academies, which is near the town of Gwoza, captured earlier this month.
A picture taken on August 21, 2014 shows Internally Displaced People (IDP) standing outside a classroom they found shelter in, at Gulak camp in Nigeria”s northeastern Adamawa State.
Thousands have fled recent fighting – these people are now living in a school
The Nigerian soldiers are currently in the Cameroonian town of Maroua, about 80km (50 miles) from the Nigerian border, Lt Col Badjek told the BBC.
Thousands of civilians are also said to have fled across the border.
In May, some 300 people were killed in an attack on Gamboru Ngala, which left much of the town in ruins.
It is near Gwoza, the largest town under control of Boko Haram. In the most recent census, in 2006, it had a population of more than 265,000 people.
In the 52-minute video released on Sunday, Boko Haram leader Abubakar Shekau said Gwoza was now “part of the Islamic state”.
He did not specify whether his groups now had any links to the Islamic State (IS) group, which has seized much of northern Iraq in recent months, prompting the US to respond with air strikes.
There is no evidence for such links but in July, Mr Shekau congratulated IS on its territorial gains.