An empty glass bottle has to be carried away from its place of use somehow. It can be taken back in the same lorry that was used to transport the full bottle, or that lorry can return to its depot with a cargo of air, and the empty bottle transported to a landfill by another lorry.
I don't know anyone who still gets milk delivered, but here in Germany the common way to buy beer is in glass bottles by the crate. A half litre bottle bought this way is somewhere between 60 and 90 cents for the contents. There are 8 cents deposit on each bottle and €1.50 on the crate itself. It's hard to believe that not returning beer bottles is more cost effective than returning them. Milk, were it still delivered, only adds one more leg to the supply chain. Either way the only issue seems to be whether washing and reuse plus the slight increase in personnel cost outweighs the total cost of more conventional disposal. Any remotely efficient washing operation should consume far less energy than the manufacture of glass from scratch. Reuse after all also eliminates the need to supply raw materials to the factory and the cost of transport from factory to filling depot.
The reduction in the amount of dairy farms would seem to be only distantly related. The number of dairy farms will increase or decrease to match the demand for milk at the price paid. If households are using milk more efficiently (wasting less), or relying more on UHT products which spoil less quickly this will affect demand. I can imagine it's easier to budget for your milk use with the weekly supermarket run and topping up if necessary from the corner shop than remembering to set the pint count thingy on your milk basket every evening.