Select the label whose color you want to change and adjust its "Border" or "Background" property.
You can select an individual label either by clicking on it or by using "Tools" -> "Select Elements". If clicking on a label selects its owner node instead, press and hold CTRL while clicking. This will cycle the selection through all the elements below the mouse cursor.
To change a property of a selected label, either use the properties view in yEd's lower right corner or press F6 to open yEd's properties dialog.