In your number manager, you'll see the option to "deactivate" or "release" each number. If your plan is to eventually release some numbers, deactivate them first to allow them to cool off.
Why is this important?
When a number is released, the line of communication between you and all prospects assigned to that number is cut. This is a permanent action, as this number no longer belongs to you and goes back to the carrier. If someone replies to a released number, you won't receive the message.
Deactivating a number before releasing the number will remove it from the pool of numbers used to send Bulk Messages, but will still allow you to receive messages from people who you are already talking to and respond to them with that number one-on-one in the prospect page. Eventually, you'll notice the number has not been recently used, and you should be fairly safe to release it.