That way madness lies
There was a website a while back called "Beat Paths", that sought to rank teams purely based on who beat who, and tried to undo all the loops that creates in some way. (I don't remember what their algorithm was. I think in some cases it just ignored the results. Like if teams A, B, and C all went 1-1 against each other, none of those games got counted.)
It actually did a pretty good job I think, but would also come up with some real oddities when there were obvious upsets.
Here's another one I've had some "barstool arguments" about. In the early 90's, FSU, and Notre Dame were both highly ranked. Notre Dame beat FSU. But then later Notre Dame also lost to some also-ran, like Boston College. My contention was the FSU was the better team and should go to a bowl ahead of FSU, even though the had the same record and Notre Dame beat FSU. My rationale was that if they both only have 1 loss, the team that had the better loss should be ranked higher. (I.e. ignore head-to-head and just look at how good their loss was).