Because the NFL would be taking away the home game from the #1 seed.
If that's what you believe, the appropriate remedy is to
give the #1 seed their home game back, not to play the game at a neutral site.
The problem is that who the #1 seed should be is unclear... but the answer to that problem is not to rewrite the rules on what constitutes a neutral game. For over 50 years, it has been determined that between the two conference champions, we ignore their records and play the game at a neutral site. No effort is made to ensure parity in the crowd, even if this disparity benefits the team with a worse record. This is the way "neutral site" games in the NFL work.
But you're saying this rule needs to be changed this one year, for this one game, which doesn't make sense. Either change it for all neutral site games going forward, or leave it alone. It's absurd to say that it's
unfair for 90,000 Bills fan to be at the "neutral site" AFC Championship Game, but then
totally reasonable for those same 90,000 Bills fans to be at the "neutral site" Super Bowl two weeks later.