Can anyone recommend a good tool for simulating the operation of a schematic made up of components such as PLLs, VCOs, mixers, filters.

Thanks

 

Sounds like something LTspice could accomplish. The main issue with a "black box" form of simulation is that it is difficult to create a general component that is capable of simulating any and all manufactured variants available. Instead, it is much more practical to choose specific components that you intend/desire to use in a circuit and simulate them in a piecemeal fashion. For example, to create a mixer, you could use LTC5510 with the necessary passive components, and since this is a linear tech IC, there should be a LTspice model available.