Downing Street has said that talks with France are in their "final stages", but the Immigration Minister has been forced to defend the Government's handling of the asylum system, saying that numbers being held in a centre in Kent were now coming down.