I find your comment very interesting, because I view it somewhat the opposite. Basically, many take your statement just the way you ordered it:

#1 Take care about your own family and children #2 Do not harm others

Then, it becomes very easy to cause harm to others and justify it through the idea that one's first priority is taking care of your own family and children and that outweighs the harm you are doing to others.

Which is just to say, it's not that simple.