Father's right to refuse to let baby breastfeed?

Long story short- was in a short, abuse and controlling relationship last year, got pregnant. My son just turned 5 months old. The father, wants full control of the baby, has told me (when the baby was three weeks old) that the baby should not have any preference to me, the one who carried him for almost 10 months and 100% breastfed him since birth. In short, he treats the baby like a piece of property to be won. He's suing me for visitation. He wants every other weekend, etc. We live 3 hours away. The baby is 90% breastfed- just starting solids. The baby cannot/will not drink from a bottle- I've tried! His father has been on my case since he was three weeks old to wean him to a bottle with formula so that we can be "equal parents and I'm not the baby's favorite". He tells me I must force the baby to learn to drink from a bottle (if he really knew the baby, he'd know that this baby would rather starve!) and that once the baby's on the bottle, he will feed the baby formula, of his choice, even throwing away pumped breast milk, because it came from me. The baby does have a milk allergy (I have to avoid eating dairy or he gets colic), and possibly other food allergies, so this will be an ongoing issue with the baby starting baby food, as well as the baby being unable to drink most (all?) formula. Does the father have the legal right to choose what the baby is fed while in his care, or to refuse to feed the baby breast milk because the mother provided it?