A straight line edge is a polyline edge without bends. Make sure your default edge style is one of the two available polyline styles. I.e. in the "Edge Types" palette section either the first or the second style has to have a blue background:
Then follow the Getting Started guide for creating edges. If you accidentally create a bend in the edge, either finish edge creation as usual, then select the undesired bend, and type the DELETE key to remove it or right-click during edge creation to remove the last bend instantly.
Moreover, you can convert any existing edge into polyline edge by selecting the edge, then right-clicking one of the polyline styles in the "Edge Types" palette section to open the corresponding context menu and choosing "Apply" or "Apply Type". After converting the edge to a polyline edge, undesired bends may be deleted by selecting the bends and typing the DELETE key.
Finally, if you have many polyline edges with bends that you want to convert to straight lines, you can use "Layout" -> "Edge Routing" -> "Straight Line" to do so.