I certainly agree about Carcer ,if only because he actually BELIEVES that he is in the right...TP quite clearly tells us that Carcer has decided that morality does not count for anything and therefore that he (carcer) only has to consider Carcer..i.e that HE is the only person that has any rights.....this is why he truly believes that he is innocent , after all , if only he has rights then no crime has been commited
Another contender has to be Teppics` chief councillor in Pyramids , a man that hog ties an entire civillisation for no better reason that "this is the way that it has always been done , so this is the way that it will always BE done"
Personally I find some of Pratchetts` concepts more frightening than his villains , and the auditors have to lead the list here , they ARE alive (by most definitions), yet they insists that they are not , and in common with carcer have put themselves above the entire multiverse as a result
A very close second place must go to the ( & let`s not bandy words here )...ELVES , who seem to have a lot in common with Dr Who`s the Silence , as both races can manipulate your mind to make you believe that that they are not there....a very hard enemy to fight.
On a more personal level I have to add that to my mind the most frightening monsters of all are the Weeping Angels from Dr Who , Pratchett has yet to come up with a villain who you know will get you because of your own weakness ( I am sure that we ALL tried not to blink after the first time that they appeared)
Incidentally I totally agree with earlier replies suggesting a story around hex (or a thinking engine like him) being based on 2001. Perhaps a ship "manned " by Vampires ? After all , they don`t need oxygen/food/water until they get there and when they do get there , can be brought to life with a vial of blood