Chancellor Angela Merkel said Monday that she would talk with all mainstream parties about trying to form a "good, stable" government after Germany's watershed election, and vowed to try to win back voters who supported an upstart nationalist force.

Sunday's election saw the rightwing populist Alternative for Germany (AfD) party poach one million votes from Merkel's conservatives, leaving her without an obvious coalition to lead Europe's largest economy.

"We had hoped for a better result," she admitted, referring to her CDU/CSU bloc's score of 33 percent, its worst outcome since 1949.

Merkel, 63, said she would now seek exploratory talks on an alliance with two smaller parties, the pro-business Free Democrats and the ecologist Greens.

And she said she would extend an olive branch to the Social Democrats, her junior partners for eight of her 12 years in power, who suffered a crushing setback with just 20.5 percent share of the vote and pledged to go into opposition.

The vote marked a breakthrough for the anti-Islam AfD, which with 12.6 percent became the third-strongest party, and it vowed to "go after" Merkel over her migrant and refugee policy.

Merkel herself acknowledged that she had been a "polarising figure" to many people who ultimately gave their vote to the AfD, noting that voters in the AfD's strongholds in depressed corners of the ex-communist east felt "left behind".

German elections
She said she believed that not all were diehard supporters of the AfD and that at least some could be won back "with good policies that solve problems".

News weekly Der Spiegel said Merkel had no one but herself to blame for her election bruising.

"Angela Merkel deserved this defeat," the magazine's Dirk Kurbjuweit wrote, accusing her of running an "uninspired" campaign and "largely ignoring the challenges posed by the right".

-'Invasion of foreigners' -

The entry of around 90 hard-right MPs to the glass-domed Bundestag chamber breaks a taboo in post-World War II Germany.

Chancellor Angel Merkel is seeking exploratory talks on an alliance with two smaller German parties, the pro-business Free Democrats and the Greens
While joyful supporters of the AfD -- a party with links to the far-right French National Front and Britain's UKIP -- sang the German national anthem at a Berlin club as the results came in late Sunday, hundreds of protesters outside shouted "Nazis out!"

The AfD's top candidate in the election, Alexander Gauland, told reporters Monday that the party was the one true defender of a Germany for the Germans.

"I don't want to lose Germany to an invasion of foreigners from foreign cultures," he said.

He refused to back away from recent comments urging Germans to be proud of their war veterans, and calling for a government official who is of Turkish origin to be "dumped in Anatolia".

But just hours after its triumph, the party's long-simmering infighting between radical and more moderate forces spilled out into the open at a dramatic news conference.

Frauke Petry, co-leader of the rightwing AfD, stunned her colleagues by saying she would not join the party's parliamentary group and would serve isntead as an independent MP
The AfD co-leader Frauke Petry stunned her colleagues by saying she would not join the party's parliamentary group and would serve as an independent MP. Another leading figure in the party, Alice Weidel, accused Petry of "irresponsibility" and urged her to quit.

Political scientist Suzanne Schuettemeyer of Halle University in eastern Germany said the AfD's presence in parliament would harm the country's image abroad.

"It's Germany and it will change the way we are perceived, because AfD will speak a language that we thought... was outside of our political consensus," she told AFP.

- 'Bitter disappointment' -

All other political parties have ruled out working with the AfD, whose leaders call Merkel a "traitor" for allowing in more than one million asylum seekers since 2015.

Merkel said that while she was not seeking a repeat of the influx, she stood by her decision made on "humanitarian" grounds.

But the leader of her Bavarian CSU allies, Horst Seehofer, a vocal critic of Merkel's asylum policy, called the vote outcome a "bitter disappointment" and pledged to close the "open flank" on the right before state elections next year.

The Social Democrats' leader Martin Schulz, putting a brave face on his defeat, said the 150-year-old party, traditionally the voice of the working classes, would be "a strong opposition force in this country, to defend democracy in this country against those who question it and attack it".

- 'Shape continent's future' -

Protesters gathered to voice their opposition to the hard-right AfD, which is now Germany's third-largest parliamentary party
This will probably force Merkel to team up with two smaller, and very different, parties to form a lineup dubbed the "Jamaica coalition" because the three parties' colours match those of the Caribbean country's flag.

One is the liberal Free Democratic Party (FDP), which with 10.7 percent made a comeback after crashing out of parliament four years ago.

The other is the left-leaning Greens party, which won 8.9 percent on campaign pledges to drive forward the country's clean-energy transition.

But with marked differences on issues ranging from EU integration to immigration, months of horse-trading could lie ahead to build a new government and avert snap elections.

European Commission chief Jean-Claude Juncker urged Merkel to form a stable government as soon as possible.

"Europe needs a strong German government now more than ever, one able to actively shape the future of our continent," he said.