The only down-side to a wood handle, really, is that it can be prone to expansion and contraction. That's because wood is porous -- that is, it has natural pores and passages.
If it didn't, it wouldn't be wood, would it?
Wood used for modern knife handles is thoroughly dried (often in a kiln) before it's cut into slabs and shaped. Some of the softer and less-dense types of wood are also "stabilized."
The stabilizing process involves impregnating the wood with industrial resin. The objective is to fill any voids and pores with material that's unlikely to be affected by changes in temperature and humidity. It also goes a long way toward making the wood tougher.
Even the best stabilized wood, however, still can expand and contract -- it may not, but if it does it's simply a characteristic of the material. Knifemakers do their best to prevent it, but it can happen.
The good news is that it's truly not a problem.
If you have a wood-handled knife and notice that either the tang or the handle is slightly "proud" (contraction or expansion of the wood, respectively), you have a few options. The first, obviously, is to acknowledge that it's simply the nature of the wooden beast and accept it. It'll probably expand or contract back into place eventually anyway.
An alternative would be to contact the knifemaker and have the handle re-finished. Most reputable 'makers will do this work under warranty at no charge to you. If you go this route, keep in mind that the "fix" may be only temporary -- wood is still wood.
Your third option, if you simply can't abide wood's quirks, would be to buy knives with handles made of synthetic materials like Micarta, G-10 or acrylic.
In the end, you'll need to decide if wood's unique beauty is worth the possibility that it may grow or shrink a little bit.
For us, it's definitely worth it.