godot: Editor crashes when clicking on empty area in 2D scene (GCC 6)

Hi!

Operating system or device - Godot version: Arch Linux, GCC 6.1.1, x86-64 build, Godot 2.0.3

Issue description (what happened, and what was expected): Editor crashes when clicking on empty area in 2D scene.

Steps to reproduce:

• compile Godot “target=release_debug” using GCC 6 (any optimization level other than -O0),
• run the editor,
• switch to 2D scene,
• add root Node,
• click on empty area.

Current behavior: crash Expected behavior: doesn’t crash

I’ve noticed that Godot calls a method cast_to() when the pointer points to 0x0. This method doesn’t use member variables, but anyway it crashes on GCC 6. I think calling a method when pointer is 0x0 especially for casting from this is not a good idea, because the behavior is undefined - in most cases it works properly, but - as You can see - not always (e.g. GCC 6 or probably other special cases).

Godot crashes somewhere inside C++ library __cxxabiv1::__dynamic_cast (the rdi register - which is first function argument treated as a pointer - is 0x0, so it must crash):

        50 [1]  in /build/gcc-multilib/src/gcc/libstdc++-v3/libsupc++/dyncast.cc
0x7f42c08e89d0  <+0x0020>        48 8b 07                    mov    rax,QWORD PTR [rdi]

When compiling with -O0 optimization the it works fine - it doesn’t call the __cxxabiv1::__dynamic_cast function and it returns NULL immediately. The __cxxabiv1::__dynamic_cast function mustn’t be called with NULL argument. It is difficult to debug it on -Og level due to optimizations (can’t debug on -O0, because it works properly), but when I reading the assembly code, it looks for me like GCC 6 does an optimization: it doesn’t check the this argument before calling the dynamic_cast, because it expects that this will never be NULL.

This patch fixes the issue:

diff --git a/tools/editor/plugins/canvas_item_editor_plugin.cpp b/tools/editor/plugins/canvas_item_editor_plugin.cpp
index 0213dbd..7f045e9 100644
--- a/tools/editor/plugins/canvas_item_editor_plugin.cpp
+++ b/tools/editor/plugins/canvas_item_editor_plugin.cpp
@@ -1492,7 +1492,7 @@ void CanvasItemEditor::_viewport_input_event(const InputEvent& p_event) {
        while ((n && n != scene && n->get_owner() != scene) || (n && !n->is_type("CanvasItem"))) {
            n = n->get_parent();
        };
-       c = n->cast_to<CanvasItem>();
+       c = n ? n->cast_to<CanvasItem>() : 0;
 #if 0
        if ( b.pressed ) box_selection_start( click );
 #endif

The alternative is to check whether this isn’t NULL before calling the dynamic_cast inside Object::cast_to<>() method.

Maybe some other crashes caused by GCC 6 are also related to this?