I managed to solve the problem, without really being able to completely explain it.
Here is how I found out :
I deleted all the girls in my first brothel : problem still there
Reloaded, deleted all the girls from my 2nd brothel : problem solved
I continued and narrowed it down to 1 girl (security job, Charming, Small Scars, Abnormally Large Boobs), Jessica Lynn
I deleted all her inventory : problem solved
I narrowed it down to 1 item : Rangiku sword (granting the Abnormally Large Boobs Trait).
I left the items on Jessica Lynn and assigned her to an other job : no bug.
I put the sword on a girl with Small Scars, on a security job : no bug.
I put the sword on an other girl with Charming and security job : bug again
I removed the sword (put it in my inventory) and gave Abnormally Large Boobs (with an Amazing Boob Job consumable item) to an other girl with Charming on security job : bug.
So, I thought it was the Charming + Abnormally Large Boobs + Security job combination that caused the bug, but I tried to give the sword to an other girl with Charming on security job and no bug.
So, I solved it, but can't explain exactly what caused it.