My Husband Ex Keeps Showing Up And His Family Prefers Her

I have been with my husband for a year and a half, and he is a good man and a good father. My only issue is his ex who still comes over to the house. His mother tells me that my husband's ex will always be a part of her life because they were together for two years.

We have a child together and we are married, and my mother inlaw doesnt see anything wrong with that
We recently stayed at my  mother inlaw's place while our place was being renovated and his ex was coming over at odd hours of the night, like 3:00 am. I’ve argued with my mother inlaw over this because it keeps me uneasy knowing my husband is in the living room in his boxers and she in the house.
His siblings say they would never choose me over her because she is their sister.I am so confused right now.My husband has not said anything about this.