Depends on how you value your integrity, and whether you're prepared to sacrifice what you have for what may potentially be a 1 off disaster. How strong is your relationship? And how much would losing it cost you?
How would you cope if they fucked others behind your back, despite your strongly voiced objections?
Have you spoken to your partner about what your reservations are?
Is it a threesome with a particular third party you have your eye on, or a specific invitation you've had, or just the general idea?
Threesomes can be fantastic like
porn, in any combination, but far better you grow into it together.