My spice cabinet includes sumac, pebrella, Mexican oregano, and three kinds of smoked paprika, but I don't have any black pepper. It's not a distaste or anything, I just never seem to get around to buying any - I haven't got a pepper grinder and keep putting off buying one I'll like. But you know, what I DO have gives me a lot of options.
This isn't an ad or anything, but I have to praise The Spice House. A lot of people are familiar with Penzey's, and order their spices there - Penzey's is certainly excellent, and I've got nothing bad to say about them. I originally placed an order with The Spice House - owned and run by members of the Penzey family, but I don't remember the exact relationship - because they carried cassia buds, which Penzey's didn't and which I wanted for a bitters recipe. I ordered other things because I always order spices in batches, in order to offset the shipping cost, and I've stuck with the Spice House ever since.
For one thing, they carry a lot of things that you not only won't find in your supermarket - I know Penzey's fans usually become aware of Ceylon cinnamon pretty quickly, for instance, and one of the things I like from both stores is crushed jalapeno peppers - little flakes like the crushed red peppers you put on pizza, only jalapeno. Online, it's also easy to find things like arrowroot, an assortment of curry powders, file powder, fennel pollen, saffron, epazote, grains of paradise, poppy seeds, and Sichuan peppercorns. Weirdly, while other areas of the supermarket expand, I've noticed the spice selection at my local supermarkets is remarkably small, concentrated on a just a couple brands and actually offering a smaller variety of spices than they carried twenty years ago when I was shopping in the same damn town. Whether this is related to the enormous growth of bottled marinades and the condiment boom of the 1990s is left as an exercise for the reader or graduate students in local culinary history.
When it comes to those slightly uncommon items, or even the same items you see in the supermarket, you're going to find higher quality online - whether you're looking at The Spice House or Penzey's - because everything is fresher. But you're also going to save money. The other day I was out of ginger, so I grabbed some in my normal supermarket run - one of the little jars the size of a Zyrtec container. Cost me six dollars. That same amount in a glass jar from The Spice House would be $3.50 - in a reclosable bag, $2.50.
It's just like with my coffee: I spend a lot on a coffee order, but that's because I buy it 10-15 pounds at a time, and I'm saving nearly as much as I'm spending, and getting better coffee in the process.
But the other thing is how much The Spice House has that hardly any other spice vendors have. The cassia buds I already mentioned. Truffle salt is another thing I pick up sometimes. Citric acid, which previously I had to buy from wine or specialty chef stores, which meant a separate shipping charge. Molecular gastronomy components like sodium alginate, xanthan gum, and some of the Texturas line. And they've started carrying black garlic, which there were only a few online vendors for before, which always meant paying a separate shipping charge. All of this stuff is lightweight, so you're almost always going to be paying $10 or less for shipping on a large order, and the savings absorbs that easily.
(This is why I sometimes go without ginger or cinnamon for a long time, though - I'm waiting to run out of other things so I can place an order.)
Like I've said a bunch of times, the quality of your cooking mostly comes down to the quality of your ingredients and your ability to use them to your best potential - that's why I've become so dependent on places like this.