The first thing to do is to figure out what they’re swarming around and get rid of it or put it in the fridge. Once you’ve done that, putting out a trap can help get rid of the fruit flies.
I got this tip from a friend some years ago. Pour a little water and apple cider vinegar into a jar or other container, and add a drop of dish soap. (Usually a little less than a tablespoon each of water and vinegar.) That’s it! Then just leave it undisturbed; I usually put it at the back of the counter. (I speculate that leaving it near a wall is good because it reduces fast air movement around the jar.)
The flies are attracted to the scent of the apple cider vinegar, and once they touch it they get stuck because the dish soap lowers the surface tension of the liquid. If there are a fair amount of flies around, I find that if I leave the trap for about 30 minutes, there will be some stuck flies once I come back. After about a day or two, most of the flies will be trapped.
Note that you won’t be able to get rid of flies if you did not get rid of everything that they are happily eating. Also, check to make sure there are no larvae or pupae on the underside of the lid of your compost container. (Once eggs are laid, they take a bit over a week to mature, so the fly population can bounce back after you think you’ve gotten rid of them.)
The vinegar in the trap loses its fly-attracting scent after a day or two, so if you still have flies, you should change the vinegar.
In writing this post I did a little Google searching to see what other people do, and, wow, there are a lot of DIY methods! I like this vinegar trap because it seems like the least effort, and it’s been effective.