Transport Secretary Mark Harper has accepted delays to HS2 will not save money.
He told MPs the Government had to make “sensible decisions” due to having annual budgets.
It was announced in March that construction of the Birmingham to Crewe leg of the high-speed railway will be delayed by two years and services may not enter central London until the 2040s.
“But of course it does reflect the fact that you have a budget in each year, everybody listening to this has to live within their annual budget, as well as a budget over time.
“We’ve had to make some sensible decisions about how you make those decisions. For me, the choice was very clear.
“There’s a section of the project which we have fully mobilised. We’ve got contractors building it, spending a considerable amount of money on phase one (between London and Birmingham).
He added: “Delaying the later aspects of the project, it’s much more cost effective to do that because you don’t have contractors on site, you aren’t already building the railway, and you can change the timing of that in a much more sensible way.”
A budget of £55.7 billion for the whole of the project was set in 2015.
But the target cost excluding the eastern leg of Phase 2b from the West Midlands to the East Midlands has ballooned to between £53 billion and £71 billion (in 2019 prices).
Mr Harper said he was “very clear” with the board of HS2 Ltd about the need to “deliver the project to the budget that has been set”.
The high-speed line was due for extension between Birmingham and Crewe between 2030 and 2034 to help boost transport in the north of England.
The Cabinet minister said: “It was significantly ahead of the budget that is available and that’s why I’ve taken the decision to pause construction of that project.
“There is going to be some cost involved in demobilising the work there and leaving the site in a sensible way for the next couple of years that is sensible for people who live in the area.
“Then we’re going to go back and look at coming up with a more cost-effective design for Euston.”
He added: “The commitment to take HS2 to Euston remains in place and the timing of that will mean that that is delivered when we have the line open to Manchester, and my understanding from having interrogated officials is that’s when the volume of passengers will require the service to go all the way to Euston.”