I seriously don't think that will happen. Here's why:
#25 Florida State hosts a terrible Pitt team - They will win that one
#24 South Florida plays a terrible North Texas team - they will win
#23 Memphis is on a bye
#22 Cockeye State plays a terrible Colorado team
#21 Arizona State plays at Utah - Arizona State could lose that game. Well guess what? If Arizona State loses to Utah, Utah is going back into the top 25. They're already receiving votes over Nebraska.
#20 Vanderbilt is on a bye.
#19 Virginia is on a bye.
I do think there is a possiblity #17 Illinois drops out completely after they play Ohio State. But they're also high enough in the poll that it might not matter.
Texas will beat Oklahoma and move back into the top 25. Mateer won't be back despite the stories like "Mateer Could return." That's just showmanship. Penn State probably beats Northwestern this weekend, and also jumps back into the top 25.
Would Nebraska be favored in a neutral side game right now vs. South Florida, Memphis, Virginia, Cockeye State, and even Illinois? Yes.