I think it might have been the giant’s big eyes. At least, I remember being scared of Sleestaks on the original Land of the Lost — I still get a start when I see a picture of one. Spider eyes, too. And grey aliens. And skulls. It probably all boils down to things that resemble skulls.