It took me a little while myself to come to terms with how I feel about speaking German with my children when we were in English-speaking company. It can feel a bit like whispering, which is regarded as highly impolite in company. After all, you might be talking about someone who is present. How would they know?
Then I read an article about bilingual families, which said that speaking a different language with your children than with everybody else should in fact not be seen as impolite as it is connected with educating children, which is of great importance and should never be undermined. It wouldn’t be unrealistic either to think that the people who are close to you, like friends, in-laws, neighbours, etc., are not likely to think you are exchanging impolite comments about them in a language they don’t understand, unless you do that in English as a habit, which would render all other consideration of politeness obsolete. People who know you also know your usual level of politeness and should have no reason to believe that when you change languages, you change in character, too. Keep in mind that how you speak also conveys a lot about the mood of what you are saying, even if the words are not intelligible to others. As long as you don’t point fingers and snigger, there is no reason for anyone to be offended.
After letting this way of thinking settle for a bit, I started to feel much more comfortable speaking German with my children in public. What I also found helpful in making myself and everyone else comfortable with me speaking two different languages in company is to translate what I said in German for whoever is with us. Most of the time it is enough to give a summary or even just the topic of what we are saying, just to keep the other person or people involved and give them the chance to take part in the conversation. Often conversations develop that we can continue in both languages without anyone feeling left out. It is important, however, that every time you address your children, you address them in your language and speak the other language (English in my case) when addressing the other people in your company. That way you are avoiding confusion in your children about what language they are expected to speak.
Another note on the sidelines. My children are three and seven years old now and are speaking German fluently, maybe with a slightly outlandish grammar at times, but they are also speaking German to each other. When their teachers or other people hear them speak German who are not used to it, they always comment on how lovely it is to hear them speak German. So really there is no reason to feel self-conscious about it. For the kids this is a completely subconscious process. It would have never been possible if I had spoken English to them in certain contexts. While children are extremely clever in terms of language acquisition, they do not have the mechanism that we adults have to consciously swap between languages depending on the situation. They can only connect a language to a person, not to a situation, so don’t overestimate them in this way. Be the person that speaks your mother tongue to them and who expects them to speak it back to you.
So be a proud speaker of your mother tongue, whatever it may be, and pass your pride on to the next generation. Good luck!