Cemetery and graveyard share the same meaning, though cemetery, from the Old French cimetiere, appears in our language far earlier and is now far more common; although more Germanic in origin, graveyard does not see written use until the 1600s. While some sources like to claim that a graveyard belongs within a churchyard while a cemetery does not, little etymological proof thereof exists. In fact, none of my usage sources bother to compare these two terms, and all etymological sources treat them as interchangeable. Use the word whose rhythm and sound better matches your intention.
For the curious, grave, the noun, and grave, the adjective, do not share common ancestry, but grave, the noun, and grave the obsolete verb (and possibly groove) do: The noun (burial place for a corpse) comes from the inferred Proto-Germanic *grafa-/graba-; the adjective (serious, somber) comes from the Latin gravis (heavy); the verb, obsolete except as engrave, also stems from the Proto-Germanic, though it took a different etymological journey into modern English. Groove (long and narrow cut), from Germanic-Nordic, likely stems from the same Proto-Indo-European root as the Proto-Germanic versions mentioned above. Groovy, huh? And, if that isn’t enough, the other verb, grave (to clean a ship’s bottom), probably comes from a dialectical rendition of the French grève, or gravele (sand, seashore, gravel), or from the Celtic *gravo-/graw-, and is unrelated to all the other graves.
(An asterisk in etymological writing indicates a hypothetical term—its existence has been inferred rather than found.)