Thomas de Maiziere said refugees could "not choose" their host countries and called on other EU states to do more.
Trains between Germany and Austria have been suspended for 12 hours.
Germany's vice-chancellor has said the country is "at the limit of its capabilities" as more than 13,000 migrants arrived in Munich on Saturday.
Germany expects 800,000 migrants to arrive this year.
"The aim of these measures is to limit the current inflows to Germany and to return to orderly procedures when people enter the country," Mr de Maiziere told a news conference.
He gave no details. The move goes against the principle of the Schengen zone, which allows free movement between many European countries. However, the agreement does allow for temporary suspensions.
Germany's rail service Deutsche Bahn said train services with Austria would be stopped until 03:00GMT on Monday.
Politically this is a shrewd move by Thomas de Maiziere. His announcement comes just a day before he travels to Brussels to meet other EU interior ministers to discuss the migrant crisis. The measure will help him put pressure on other European countries to do their bit. It highlights just how much Germany is struggling to cope.
The move could also serve as a useful threat; after all, Mr de Maiziere said Germany was controlling the border with Austria "first", the implication being more could follow. The possibility that Germany might suddenly decide to control its other borders could well help jolt EU partners into action.
For migrants, the announcement means Germany is not pursuing an open-door policy. After weeks of confusion, Berlin is now sending out the clear message that the Dublin Regulation does still hold, meaning that people have to apply for asylum in the first EU country they arrive in. After that, if Berlin gets its way, they will then be sent elsewhere in Europe according to a strict quota system.
Lyse Doucet: Where does this crisis end?
Caring for solo child refugees
Crisis explained in graphics
What next for Germany's asylum seekers?
Many migrants have been refusing to register in countries such as Greece or Hungary, fearing it will stop them being granted asylum in Germany or other EU states.
The city of Munich, in the German state of Bavaria, has taken the brunt of arrivals over the weekend.
Bavarian Premier Horst Seehofer said the controls sent an "important signal".
Hungarian Prime Minister Viktor Orban, who has taken a tough line on the migrant crisis, told Germany's Bild newspaper he welcomed the new controls, saying they were "necessary to protect German and European values".
On Sunday, the Czech Republic also said it would boost border controls with Austria.
Europe as a whole is struggling to deal with an enormous influx of people, mostly from Syria but also Afghanistan, Eritrea and other countries, fleeing violence and poverty.
On Sunday, Greek coastguards said at least 34 people, including 11 children, drowned when a boat carrying about 100 migrants capsized off the island of Farmakonisi in the southern Aegean Sea.
The BBC's Lyse Doucet in Greece says it is the largest loss of life in a single incident in the Aegean since the crisis began.
Earlier on Sunday, Germany's Vice-Chancellor Sigmar Gabriel, who is also economy minister, warned the country was being stretched to its limits by the new arrivals.
"It is not just a question of the number of migrants, but also the speed at which they are arriving that makes the situation so difficult to handle," he told the Tagesspiegel newspaper.
Mr Gabriel also called on European countries, Gulf states and the US to give billions of euros towards schools, accommodation and food in refugee camps in the Middle East.
A steady stream of migrants is travelling from Greece, through Macedonia, Serbia and Hungary, to Austria and Germany.
Hungary is aiming to complete a four-metre-high (13ft) fence along the border with Serbia by 15 September, when tougher measures, including arresting illegal immigrants, come into force.
The European Commission announced plans last week for mandatory quotas to share out 120,000 additional asylum seekers among 25 member countries.
Hungary, the Czech Republic, Slovakia and Romania are opposed to this.
Germany is to introduce temporary controls on its border with Austria to cope with the influx of migrants, the interior minister has said.