Generally speaking, you are not required by any moral code to boycott any business because of how they choose to spend their money. If they sell morally good products or morally neutral stuff, it is not your concern how they choose to spend their proceeds.
But in your particular situation, you and a small group of people might use this as a tool to help encourage businesses to stop supporting morally objectionable practices. This is what a lot of people do when they boycott one company or another. But the practicality of this is hard to justify i.e., if you start boycotting every business that supports abortion and other anti-human or immoral practices. You would wonder, where would you shop? So, it is up to you to decide how to respond, but those responses must be honest and peaceful. No moral obligations.
