There will be cases where you may overdo it as far as wanting more peripherals installed on your motherboard as much as possible. In fact, you can fill up those slots as much as you can. But the problem starts to creep in when you feel that some parts are installed but not used. The normal thing a PC user would do is remove them but what about the slots or the holes that they would leave.
Normally, cases have a patched back where slots are opened without any provisions for patching them up. This is dangerous since rats or other forms of dust and objects may make their way into your computer. In the case of rats, if they should piss or bite some wires, you can expect some PC malfunctions to occur and that is entirely something uncomfortable.
As a precaution, it would be wise to perhaps tape them for now. Or better yet, if you can get hold of perfect fitted metal plates to shut them from intrusion of any sort, then the better. Some desktop manufacturers have this provision while others have the remove and then install only options. Maybe if you ask a friend or a technician who may have these metal blockers, then you can ask it from them since they usually throw them away if not used.
You may call it as a form of making your PC open to potential threats. Do not wait for it to happen. Use tape for now but if you can get good metal blocker plates, the better it is for you and your PC.