As of yEd 3.18.1, the yEd installer comes with the latest version of Java 10 (previous installers did come with Java 8) to support Windows display scaling (which is typically used on HDPI displays). Unfortunately, starting yEd through the yEd.exe created by the installer sometimes result in the reported problem.
Update July 18th, 2018:
This error is caused by incompatible Java versions in the Windows PATH environment variable (typically a Java 8 bin directory).
Possible solutions for yEd 3.18.1 are:
Remove the entry that points to the bin directory of your JRE/JDK installation from the Windows PATH environment variable.
(With Java 8 such an entry is not necessary anymore because the Java installer will add C:\ProgramData\Oracle\Java\javapath to PATH which is safe to use.)
Run yEd on Java 8 instead of Java 10 if you do not need support for Windows display scaling. A yEd installer with embedded Java 8 runtime environment is available from the yEd download site (click on "Show all versions") for this purpose.
Start yEd manually instead of using the yEd.exe.
Assuming you have installed yEd to C:\Program Files\yWorks\yEd, start a command prompt (you can do so by opening the "Start" menu, typing "cmd", and starting the corresponding app) and use the following command:
"C:\Program Files\yWorks\yEd\jre\bin\java.exe" -jar "C:\Program Files\yWorks\yEd\yed.jar"
This problem will be fixed with the upcoming version 18.104.22.168 of yEd.