. . .
. . .
. . .
. . .
. . .
. . .
This is a common way to collect and your best bet is to get a copy of the Standard Catalog of World Paper Money, Volume 2 (General Issues). This way you have a road map of current and past paper money and how much it will cost. Also it allows you to see which notes you really want to have which you wouldn't otherwise know about (e.g. crossed out portrait of the Shah of Iran, Faulkland Islands notes, portrait of Albert Einstein)
A good start is to buy a pile of general world notes from a dealer (e.g. 50 different notes for 20 dollars). You can then build on this over time with selected notes.
It's probably possible to put together a set of notes from all countries for a few hundred dollars. You can probably cover half of all countries for under a hundred dollars.
. . .
. . .
. . .