Almost certainly, you have to choose one. You make that choice based on how much they'll pay and what rights they want to buy.
If one publication wants to buy "all rights" they are, in essence, saying that they want to own everything about it. You can't re-sell it to anyone else, you can't use it later as part of a larger work, nothing. They own everything about that piece now, just as if THEY had written it originally, rather than you.
If another publication wants to buy "first rights" they are saying that they want to be the first ones to publish it. After that, the rights revert to you and you once again own the piece. You can then sell the "reprint rights" to someone else.
Now, the only way you wouldn't have to choose is if you sell it to a couple of different, very small and very local publications that are willing to buy very restricted rights. So, for instance, you might sell "first California rights" to a tiny little literary mag in Barstow. They now have the rights to be the first ones in California to publish it. At the same time you could sell "first Florida rights" to a tiny little literary mag in Palatka and they would have the rights to be the first ones in Florida to publish it.
More likely than that, someone might want "first North American rights." This means that you could simultaneously sell it to another publication in Europe, or Asia, or anywhere but North America.
Realistically, though, you'll have to choose. Almost no publication is going to want to buy only the rights to publish it in their state, and if they did I can't imagine they'd pay more than a pittance. These days fewer and fewer publications are even interested in "first North American rights." What they'll probably ask for is "all rights." Try to negotiate that down to "first rights."