This may happen if you shared your calendar with some of your colleagues where FreeBusy classifies them as delegates. FreeBusy has built-in features to CC calendar delegates on your scheduling activity by default.
If you do not want this to happen, you can also turn it off.
Go to the Integrations page
Locate the desired calendar
Unselect one or more people from the list of delegates
Once unselected, FreeBusy won't auto-select them again even if your calendar provider still reports them as a delegate.