Victorian Trades Hall secretary Luke Hilakari said the returning officer at the Burwood East polling booth attempted to remove the sign but was warded off by a Liberal campaigner who threatened to call the police.
Mr Hilakari argued the signage was a "massive rort" targeting voters who did not have English as their first language. The sign is authorised by the state director of the Liberal Party in Victoria, Simon Frost.
A Victorian Liberal Party spokesman acknowledged the similar colour schemes of the two signs. Asked if the Liberal sign could be seen as official AEC advice, they said: "I can see how that might be some person's interpretation but not another person's interpretation."
The spokesman later claimed that remark was off-the-record and said the party would not comment on the matter.
An AEC spokesman confirmed the organisation had received a complaint from Labor and the AEC had responded. He said the sign in question would be fine as long as it was authorised and not located within six metres of the entrance of a polling place.
Labor had complained the sign breached section breached section 329 of the Electoral Act, which makes it an offence to publish or distribute any electoral material that is "likely to mislead or deceive an elector in relation to the casting of a vote".
A Labor campaign spokeswoman said: "This is a new low - a pathetic and dirty attempt to deceive voters because the Liberals have no policy to talk about."
The hotly-contested Melbourne seat of Chisholm is home to a large Chinese population. It was formerly held by Julia Banks, the Liberal MP who defected to become an independent and is now contesting Health Minister Greg Hunt's seat of Flinders.
Labor is hopeful of winning Chisholm along with a swag of other Liberal seats in Victoria. The Liberal candidate, Gladys Liu, was criticised during the campaign for her previous comments on homosexuality and gender issues.
The AEC also reminded voters on Saturday morning: "Outside the polling places can be busy and colourful, with candidate representatives handing out material that suggests you vote in a certain way. However, who you vote for is your decision. You do not have to accept or follow any of the material if you dont want to."