This is an interesting case: it's an example where none of the options seems quite right!
Goat's cheese seems to say "the cheese of/from A goat". That works well in my pet lambs have been greedy and eaten my pet goat's cheese, but where we mean "cheese made from the milk of goats", then it seems to suggest the milk of, specifically, only one goat.
Goats' cheese says "the cheese of/from goats", which is much closer - but then are we sure that several goats really were involved? You could keep just one goat and make cheese from her milk!
Perhaps that's why we tend just to write goats cheese (and goats milk, cows milk, etc). That can still be regarded as grammatically sound, I think, in the same way as accounts department, complaints manager - the word just tells us what TYPE of cheese (or department, or manager) is involved.
Not that I am trying to score points here but I have just had to go through the same process for "quails' eggs". I was describing a party where the only drink was Perrier Jouet champagne and the only food was quails' eggs. Nice for a while but tedious after a point, especially if you are not a big fan of fizz.
In the end, I went for s-apostrophe, on the basis that the eggs were almost certainly from a number of different quails! I suspect this would be the same for the goats' cheese.
Incidentally, the party was held under the brontosaurus at the Natural History Museum.
A learned blockhead is a greater blockhead than an ignorant one (Benjamin Franklin)
Poor peterscott123. Now he's got a recommendation for every option!
Pete, I'll try and persuade you. I think the menu describes an individual portion: a quail's egg. You would surely have spoken of a Champagne and lemon tart party, even though you might have had to provide several lemon tarts?
This is like lamb's kidneys: if a recipe calls for six lamb's kidneys, that's not twelve kidneys! It's a lamb's kidney times six, no-one would suggest it means six kidneys from one lamb. Equally six quail's eggs does not require the quail to have laid six eggs: it could be a quail's egg times six.
At a stretch, quail's eggs could even be using the hunter's plural (a hunter exemplar, note) -- the eggs of quail. I shot three pigeon. A brace of pheasant.