That's according to Ken Ehrlich, the show's longtime producer.
"The fact of the matter is, we continue to have a problem in the hip-hop world," Ehrlich told the paper. "When they don't take home the big prize, the regard of the academy, and what the Grammys represent, continues to be less meaningful to the hip-hop community, which is sad."
CNN has reached out to the three rappers for comment.
Lamar leads this year with eight nominations, followed by Drake who has seven. Childish Gambino has five nominations.
The Recording Academy has made an effort to diversify its membership amid complaints that the Grammys are not an even playing field for minority and women artists.
"We don't, as musicians, in my humble opinion, listen to music based on gender or race or ethnicity," Portnow said then. "When you go to vote on a piece of music -- at least the way that I approach it -- is you almost put a blindfold on and you listen."
CNN has reached out to the Recording Academy for additional comment.
The 61st Grammy Awards will air live at 8 p.m. ET Sunday on CBS.