The prime minister wants independent senator Fraser Anning to face "the full force of the law" over his comments the Islamic community was to blame for the New Zealand mosque massacre.
And New Zealand's representative in Australia says the Queensland senator's comments were atrocious.
The government and Labor are planning a bipartisan motion condemning Senator Anning when parliament returns in April.
However, Scott Morrison went further on Sunday, when asked if a parliamentary motion was enough, saying: "I think the full force of the law should be applied to Senator Anning."
He previously said the comments the senator made were appalling, ugly and had no place in Australia.
New Zealand High Commissioner Annette King said her country rejected absolutely Senator Anning's sentiment.
"We just condemn what he said and I'm absolutely delighted that his comments were condemned from the very top of government here in Australia and the people of Australia," she told reporters in Canberra.
However, she said it was up to Australians to decide if he was fit to be an elected representative.
A change.org petition calling for Mr Anning's expulsion from parliament has amassed almost 750,000 signatures.
"We call on the Australian government to expel this man who blames victims for their own violent deaths, and uses references to genocide to further his hateful agenda," it states.
A 17-year-old boy was arrested after allegedly throwing an egg at Senator Anning at a speaking event at Moorabbin in Melbourne on Saturday.
The teenager has since been released pending further inquiries.
He posted a video on social media saying: "Don't egg politicians, you get tackled by 30 bogans at the same time - I learnt the hard way."
Senator Anning used his Facebook page to thank those who had supported him after the incident.
He made several more anti-Islamic posts and complained about "the backlash of the establishment media and globalist politicians".
He also said Twitter had banned him from posting there for breaching the platform's community standards.
Senator Anning came to the Senate in late 2017 as a replacement for One Nation's Malcolm Roberts after he was found to have been a dual citizen and therefore ineligible for election.
While Senator Anning ran on the One Nation ticket at the 2016 poll, he quit the party minutes before being sworn in after falling out with leader Pauline Hanson.
He was roundly condemned after invoking the phrase "final solution" in his first speech to parliament - a reference to the Holocaust - and again after attending a far-right rally in Melbourne where participants made Nazi salutes.