I doubt that is possible unless you are programming the serial driver yourself and are willing to sacrifice 4-5 input lines to select which of these 3-20 devices you are using during each transmission. The recieving end of the serial line would also have to be programmed or designed to sense whether they're address is active on the select lines or not.
At least i have not heard of multiple serial for a single computer unless you are using a switch, but i think those are pretty expensive at 20 inputs, and also you would have to switch between devices manually with those kind of switches.
thanks! this is the what happen, my group present many good and quite easy proposal for a thesis like voice operated car.. but my stupid propesor make his own idea and gave it to my group.. the ida was to set-up rfid reader for every room in our university.. imagine! how many rooms we have, so we bargain it to atleast in 3-20 range.. it will cost a lot! so if we set-up atleast 3, our system must be ready for a 20 slot..