Solid-state microwave systems offer strong interactions for fast quantum logic and sensing but photons at telecom wavelength are the ideal choice for high-density low-loss quantum interconnects. A general-purpose interface that can make use of single photon effects requires,$<$,1 input noise quanta, which has remained elusive due to either low efficiency or pump induced heating. Here we demonstrate coherent electro-optic modulation on nanosecond-timescales with only $$0.16_-0.01textasciicircum+0.02$$microwave input noise photons with a total bidirectional transduction efficiency of 8.7% (or up to 15% with $$0.41_-0.02textasciicircum+0.02$$), as required for near-term heralded quantum network protocols. The use of short and high-power optical pump pulses also enables near-unity cooperativity of the electro-optic interaction leading to an internal pure conversion efficiency of up to 99.5%. Together with the low mode occupancy this provides evidence for electro-optic laser cooling and vacuum amplification as predicted a decade ago.