LSE (LNSTY) added that it sees "no merit in further engagement" because of the offer's "fundamental flaws." The London exchange said it remained committed to its acquisition of financial data provider Refinitiv.
HKEX responded in a statement that said it "continues to believe that the proposed combination ... represents a highly compelling strategic opportunity."
The Hong Kong company suggested that it could now make a hostile bid that would allow investors in LSE to choose between an improved offer and the planned purchase of Refinitiv.
"HKEX believes that shareholders in LSE should have the opportunity to analyze in detail both transactions and will continue to engage with them," the statement said.
Analysts at Citi said in a research note that they expect a second offer from HKEX that includes more cash and improved terms. But they warned that regulatory hurdles still threaten any deal.
LSE chairman Don Robert said the exchange was "surprised and disappointed" that HKEX (HKXCF) published its "unsolicited proposal within two days of our receiving it."
The unexpected bid was published Wednesday, suggesting that LSE received the offer on Monday.
The transaction posed serious risks and lacked value for shareholders, Robert said in a letter addressed to the chairperson and CEO of HKEX.
HKEX's relationship with the Hong Kong government would "complicate matters," making it "highly uncertain" that necessary approvals would be obtained, Robert said.
The Hong Kong government directly appoints half of the HKEX board, according to its website. And the chairman's appointment must be approved by Hong Kong's chief executive, Carrie Lam.
Robert also raised concerns about continued social unrest in Hong Kong, saying the "ongoing situation" in the territory adds to uncertainty.
For shareholders, the proposition was unattractive given that they would be paid mostly in HKEX shares, Robert said.
"We see the value of your share consideration as inherently uncertain," said Robert. "Furthermore, we question the sustainability of HKEX's position as a strategic gateway in the longer term."
The proposal would be a "backward step" for LSE strategically, given the high geographic concentration of HKEX's portfolio.
"We do not believe HKEX provides us with the best long-term positioning in Asia or the best listing/trading platform for China," he said, noting that LSE values its current partnership with the Shanghai Stock Exchange.
Even if the proposal were deliverable, it fell "substantially short" of an appropriate valuation for a takeover of LSE, "especially when compared to the significant value we expect to create through our planned acquisition of Refinitiv," Robert said.
LSE's forceful rejection was aimed at discouraging HKEX from pursuing the deal any further, rather than dissuading a third party from getting involved, Chris Turner, an analyst at Berenberg told CNN Business.
Berenberg has previously highlighted CME and Intercontinental Exchange, which owns the New York Stock Exchange, as other potential suitors. ICE had previously mulled a bid for LSE in 2016.
Pamela Boykoff contributed reporting.