1) Materials: Depending on how long you will be locked up, you will need to consider what the device is made of. Metal ones tend to be more durable, longer, but will set off metal detectors if you travel or have to go into "secured" buildings. Plastic/resin moulded devices are less likely to be a nuisance (depending on how they are secured), but require special care to keep from deteriorating and breaking. I have also seen "soft sided" devices made from some combination of rubber, vinyl, and polyurethane. Since part of the purpose of a chastity device is lack of sensation, I would steer clear of the soft devices.
2) Fit and form: Again, depending on the duration, you will want to pay attention to how well the device fits and how comfortable it is. Full belt models, as I understand, tend to be more comfortable overall compared to "cock locks", but require special fitting around the waist. Cock Locks (e.g. CB-X series devices, Holy Trainer, etc) are usually easily fit and are fairly effective, but take some trial and error to get "just right". Plus, they do not always prevent erection attempts while sleeping until the wearer acclimates, regardless of the advertising, so be ready for some rough sleep the first week or so. Whatever device you go with, try to get out of it when doing the first fitting. Best to find out the weaknesses early on so they can be addressed.
3) Duration: Take it slow, if you have the option. Start with wearing the device for a few hours at a time, then increase gradually- overnight, 24-48 hrs, a week, etc. Give yourself time between sessions to recover from any chafing that may occur, and be honest about the comfort level.
4) Cleaning: Pay attention to the cleaning and care instructions for the device. Is it dishwasher safe? What types of soap or cleanser is recommended? What type of lubrication is recommended, and why? Also consider what you will need to keep YOURSELF clean and cared for while locked up.
5) Cost: This one is two fold. First, understand that quality costs. You can find knock-offs of the more popular devices all over the internet. It is very much a "buyer beware" situation. Quality of manufacturing, materials, and warranty differences need to be a concern. Also, don't expect to love your first device right out of the gate. It took me three or four tries to find the one that was right for me, and the devices are non-refundable in most cases. (Seriously, would you want to wear a second-hand cock sleeve?)
6) Security vs Necessity: Plan for the worst, hope for the best. There are going to be times that you will need to be released for some reason, so plan for it up front. The CB-X series comes with numbered plastic "locks" in addition to the included padlock. The Holy Trainer "magic lock" has a standardized key. Work with your Keyholder to plan for emergency release scenarios.