Here's what really killed The X Factor after 17 gruelling years