The orbital refinery is somewhat necessary for lunar mining. But lunar mining isn't necessary for the orbital refinery. The answer, therefore, is asteroids.
<snip>
To go to the Moon with existing spacecraft, requires a means delivering rocket fuel into space. Either depots or docking of spacecraft.
In terms of lunar operations a critical location for docking or re-fueling is high earth orbit: GTO, GEO, L-points, or lunar orbit. And if one is also docking or re-fueling in LEO, one can use medium to small launchers to get to the Moon.
One could restrict it to only docking in LEO, and use existing hvy launchers- docking two payloads from two hvy launchers [though one could have multiple smaller launches and docking in LEO].
If one docking in some Lunar orbit and using hvy lift: hvy being any rocket that put about 5 of tonnes in lunar orbit. Any rocket that put 5 tons to GTO can put about 5 tons to Lunar orbit.
Existing Hvy lifts are Ariane V, Zenit, Proton, Altas V, Delta-IV.
Now, a dry weight vehicle of 5 tons is quite large when one consider that 90% mass could be rocket fuel- giving fueled mass of 50 tons- as much or more than Saturn V lifted to lunar orbit, so fuel depot is more flexible than simply docking to craft.
Fuel depot at lunar orbit or Earth/Moon L-1 essentially makes the existing hvy launch equal to Saturn V.
One could make a similar argument with depot at LEO- if have dry mass of 20 ton, it equal to fully fuel 200 ton craft [much bigger the Saturn V LEO payload.
Also one make the argument that fuel depot on the lunar surface gives a Saturn V type capability. Either of the three locations does this, and all three is giving a payload size to the moon [and returning from the moon] of around 3 times a Saturn V, to a number of existing rockets. Or about the same capability of Saturn V to medium launchers [even a greater number of existing rockets].
If you mining the Moon to make rocket fuel, this obviously is leading to fuel depot supplied from rocket fuel made of the Moon.
From a lunar miners point of view, all that is essential is a depot in L-1 with fuel on Moon being made on the Moon.
Or one start with rocket fuel delivered from Earth, to L-1 and lunar surface, and later as fuel is made on the Moon, lunar depot gets lunar made rocket fuel, followed by L-1 depot getting lunar made rocket fuel.
That combination gives greater than Saturn V capability. And lower cost because it allows the reusability of spacecraft landing and leaving the Moon.