Could be anything, and intermittent makes it hard to locate sometimes. Check all the basic things, some of which you already have. There shouldn't be too much moisture affecting it as fairly well out of the weather, even when it's pouring rain. If so, the connections aren't in great shape anyway, always possible. Apart from taking off door trims to clean terminals, not much you can do. Also, make sure your door hinges are free of rust and gunk as they form the ground for the windows. Give them a good spray with WD40 and work the doors back and forth to get it soaked through. If really gunked up you can then spray with some degreaser, and flush with hot soapy water, then only lightly oil so maintaining a good ground. You could look at replacing the relay too, see if that helps. Good to have a spare anyway in original shape that simply plugs in. Do they work ok when engine not running, other than slower?