binaryninja-api: demangle_ms bugs/improvements

Binary Ninja Version: 2.0.2138-dev, d031c340 Platform: Windows 10 Version 1903

After using demangle_ms a lot lately, here’s a list of problems I have come across:

Vftables have type_class TypeClass.NamedTypeReferenceClass

# const Foo::`vftable'
>>> demangle_ms(arch, '??_7Foo@@6B@')
(<type: const>, ['Foo', "`vftable'"])

>>> demangle_ms(arch, '??_7Foo@@6B@')[0].type_class
<TypeClass.NamedTypeReferenceClass: 11>

Multiple inheritance vftables produce invalid types and contain the parent in the type tokens instead of the name

# const Bar::`vftable'{for `Foo'}
>>> demangle_ms(arch, '??_7Bar@@6BFoo@@@') 
Traceback (most recent call last):
  File "C:\Program Files\Vector35\BinaryNinja\plugins\..\python\binaryninja\types.py", line 356, in __repr__
    return "<type: %s>" % str(self)
  File "C:\Program Files\Vector35\BinaryNinja\plugins\..\python\binaryninja\types.py", line 367, in __str__
    return core.BNGetTypeString(self._handle, platform)
  File "C:\Program Files\Vector35\BinaryNinja\plugins\..\python\binaryninja\_binaryninjacore.py", line 11021, in BNGetTypeString
    result = _BNGetTypeString(*args)
OSError: exception: access violation reading 0x0000000000000010

>>> demangle_ms(arch, '??_7Bar@@6BFoo@@@')[0].get_tokens_after_name()
['{', 'for', ' `', 'Foo', "'}"]

Certain symbols contain the attributes in their type

Attributes (static, virtual, access modifier, etc.) could really do with being returned separately instead of discarding them or making them part of the type string.

# protected: static bool Foo::Bar
>>> demangle_ms(arch, '?Bar@Foo@@1_NA')
(<type: protected: static bool>, ['Foo', 'Bar'])

>>> demangle_ms(arch, '?Bar@Foo@@1_NA')[0].get_tokens_before_name()
['protected:', ' ', 'static', ' ', 'bool']

Functions with no parameters have unnecessary void parameter

# void __cdecl Foo(void)
>>> demangle_ms(arch, '?Foo@@YAXXZ')
(<type: void __cdecl (void)>, ['Foo'])

>>> demangle_ms(arch, '?Foo@@YAXXZ')[0].parameters[0].type.type_class
<TypeClass.VoidTypeClass: 0>

Functions not given a calling convention (#1390)

# public: virtual void __thiscall Foo::Bar(void)
>>> demangle_ms(arch, '?Bar@Foo@@UAEXXZ')
(<type: public: virtual void __thiscall (void)>, ['Foo', 'Bar'])
>>> demangle_ms(arch, '?Bar@Foo@@UAEXXZ')[0].calling_convention is None
True

Non-static member functions are missing hidden ‘this’ parameter

Checking whether the symbol is a non static function with an access modifier (public/protected/private) seems to be the best way to determine if the parameter is needed.

>>> demangle_ms(arch, '?Bar@Foo@@UAEXXZ')
(<type: public: virtual void __thiscall (void)>, ['Foo', 'Bar'])
>>> demangle_ms(arch, '?Bar@Foo@@UAEXXZ')[0].parameters
[void ]

Cannot demangle member function pointers

Not sure how feasible it is to create types for these since the size of member function pointers varies.

# void __cdecl Bar(void (__thiscall Foo::*)(void))
>>> demangle_ms(arch, '?Bar@@YAXP8Foo@@AEXXZ@Z')
(None, '?Bar@@YAXP8Foo@@AEXXZ@Z')

Incorrect demangled names

# `eh vector destructor iterator'
>>> demangle_ms(arch, '??_M@YGXPAXIHP6EX0@Z@Z')[1]
["`eh vector vbase constructor iterator'"]

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 1
  • Comments: 32 (23 by maintainers)

Most upvoted comments

As of 3.6.4615, a bug in template back-references has been fixed. This has fixed the following:

??_G?$_Func_impl@P6A_NAEBW4agent_status@Concurrency@@@ZV?$allocator@H@std@@_NAEBW412@@std@@QEAAPEAXI@Z
MSVC     public: void * __ptr64 __cdecl std::_Func_impl<bool (__cdecl*)(enum Concurrency::agent_status const & __ptr64),class std::allocator<int>,bool,enum Concurrency::agent_status const & __ptr64>::`scalar deleting destructor'(unsigned int) __ptr64
LLVM     public: void * __cdecl std::_Func_impl<bool (__cdecl *)(enum Concurrency::agent_status const &), class std::allocator<int>, bool, enum Concurrency::agent_status const &>::`scalar deleting dtor'(unsigned int)
BNJA NEW void* __ptr64 __cdecl std::_Func_impl<bool (__cdecl*)(enum Concurrency::agent_status const& __ptr64),class std::allocator<int32_t>,bool,enum std::allocator<int32_t> const& __ptr64>::`scalar deleting destructor'(std::_Func_impl<bool (__cdecl*)(enum Concurrency::agent_status const& __ptr64),class std::allocator<int32_t>,bool,enum std::allocator<int32_t> const& __ptr64>* this, uint32_t)__ptr64
BNJA OLD ??_G?$_Func_impl@P6A_NAEBW4agent_status@Concurrency@@@ZV?$allocator@H@std@@_NAEBW412@@std@@QEAAPEAXI@Z

?CheckForDeletionBridge@?$ListArray@U?$ListArrayInlineLink@VWorkQueue@details@Concurrency@@@details@Concurrency@@@details@Concurrency@@CAXPEAV123@@Z
MSVC     private: static void __cdecl Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue> >::CheckForDeletionBridge(class Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue> > * __ptr64)
LLVM     private: static void __cdecl Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue>>::CheckForDeletionBridge(class Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue>> *)
BNJA NEW void __cdecl Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue> >::CheckForDeletionBridge(class Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue> >* __ptr64)
BNJA OLD ?CheckForDeletionBridge@?$ListArray@U?$ListArrayInlineLink@VWorkQueue@details@Concurrency@@@details@Concurrency@@@details@Concurrency@@CAXPEAV123@@Z

??_G?$_Func_impl@P6A_NAEBW4agent_status@Concurrency@@@ZV?$allocator@H@std@@_NAEBW412@@std@@QEAAPEAXI@Z
MSVC     public: void * __ptr64 __cdecl std::_Func_impl<bool (__cdecl*)(enum Concurrency::agent_status const & __ptr64),class std::allocator<int>,bool,enum Concurrency::agent_status const & __ptr64>::`scalar deleting destructor'(unsigned int) __ptr64
LLVM     public: void * __cdecl std::_Func_impl<bool (__cdecl *)(enum Concurrency::agent_status const &), class std::allocator<int>, bool, enum Concurrency::agent_status const &>::`scalar deleting dtor'(unsigned int)
BNJA NEW void* __ptr64 __cdecl std::_Func_impl<bool (__cdecl*)(enum Concurrency::agent_status const& __ptr64),class std::allocator<int32_t>,bool,enum std::allocator<int32_t> const& __ptr64>::`scalar deleting destructor'(std::_Func_impl<bool (__cdecl*)(enum Concurrency::agent_status const& __ptr64),class std::allocator<int32_t>,bool,enum std::allocator<int32_t> const& __ptr64>* this, uint32_t)__ptr64
BNJA OLD ??_G?$_Func_impl@P6A_NAEBW4agent_status@Concurrency@@@ZV?$allocator@H@std@@_NAEBW412@@std@@QEAAPEAXI@Z

?CheckForDeletionBridge@?$ListArray@U?$ListArrayInlineLink@VWorkQueue@details@Concurrency@@@details@Concurrency@@@details@Concurrency@@CAXPEAV123@@Z
MSVC     private: static void __cdecl Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue> >::CheckForDeletionBridge(class Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue> > * __ptr64)
LLVM     private: static void __cdecl Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue>>::CheckForDeletionBridge(class Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue>> *)
BNJA NEW void __cdecl Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue> >::CheckForDeletionBridge(class Concurrency::details::ListArray<struct Concurrency::details::ListArrayInlineLink<class Concurrency::details::WorkQueue> >* __ptr64)
BNJA OLD ?CheckForDeletionBridge@?$ListArray@U?$ListArrayInlineLink@VWorkQueue@details@Concurrency@@@details@Concurrency@@@details@Concurrency@@CAXPEAV123@@Z

??4?$_Yarn@D@std@@QEAAAEAV01@PEBD@Z
MSVC     public: class std::_Yarn<char> & __ptr64 __cdecl std::_Yarn<char>::operator=(char const * __ptr64) __ptr64
LLVM     public: class std::_Yarn<char> & __cdecl std::_Yarn<char>::operator=(char const *)
BNJA NEW class std::_Yarn<char>& __ptr64 __cdecl std::_Yarn<char>::operator=(std::_Yarn<char>* this, char const* __ptr64)__ptr64
BNJA OLD ??4?$_Yarn@D@std@@QEAAAEAV01@PEBD@Z

??0?$shared_ptr@V__ExceptionPtr@@@std@@QEAA@AEBV01@@Z
MSVC     public: __cdecl std::shared_ptr<class __ExceptionPtr>::shared_ptr<class __ExceptionPtr>(class std::shared_ptr<class __ExceptionPtr> const & __ptr64) __ptr64
LLVM     public: __cdecl std::shared_ptr<class __ExceptionPtr>::shared_ptr<class __ExceptionPtr>(class std::shared_ptr<class __ExceptionPtr> const &)
BNJA NEW __cdecl std::shared_ptr<class __ExceptionPtr>::shared_ptr<class __ExceptionPtr>(std::shared_ptr<class __ExceptionPtr>* this, class std::shared_ptr<class __ExceptionPtr> const& __ptr64)__ptr64
BNJA OLD ??0?$shared_ptr@V__ExceptionPtr@@@std@@QEAA@AEBV01@@Z

??0ObsidianBlock@@QEAA@AEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@H_N@Z
MSVC     public: __cdecl ObsidianBlock::ObsidianBlock(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const & __ptr64,int,bool) __ptr64
LLVM     public: __cdecl ObsidianBlock::ObsidianBlock(class std::basic_string<char, struct std::char_traits<char>, class std::allocator<char>> const &, int, bool)
BNJA NEW __cdecl ObsidianBlock::ObsidianBlock(ObsidianBlock* this, class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const& __ptr64, int32_t, bool)__ptr64
BNJA OLD ??0ObsidianBlock@@QEAA@AEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@H_N@Z

??4?$Resource@VTSShape@@@@QAEAAV0@ABV0@@Z
MSVC     public: class Resource<class TSShape> & __thiscall Resource<class TSShape>::operator=(class Resource<class TSShape> const &)
LLVM     public: class Resource<class TSShape> & __thiscall Resource<class TSShape>::operator=(class Resource<class TSShape> const &)
BNJA NEW class Resource<class TSShape>& __thiscall Resource<class TSShape>::operator=(Resource<class TSShape>* this, class Resource<class TSShape> const&)
BNJA OLD ??4?$Resource@VTSShape@@@@QAEAAV0@ABV0@@Z

??0?$basic_iostream@DU?$char_traits@D@std@@@std@@QAE@PAV?$basic_streambuf@DU?$char_traits@D@std@@@1@@Z
MSVC     public: __thiscall std::basic_iostream<char,struct std::char_traits<char> >::basic_iostream<char,struct std::char_traits<char> >(class std::basic_streambuf<char,struct std::char_traits<char> > *)
LLVM     public: __thiscall std::basic_iostream<char, struct std::char_traits<char>>::basic_iostream<char, struct std::char_traits<char>>(class std::basic_streambuf<char, struct std::char_traits<char>> *)
BNJA NEW __thiscall std::basic_iostream<char,struct std::char_traits<char> >::basic_iostream<char,struct std::char_traits<char> >(std::basic_iostream<char,struct std::char_traits<char> >* this, class std::basic_streambuf<char,struct std::char_traits<char> >*)
BNJA OLD ??0?$basic_iostream@DU?$char_traits@D@std@@@std@@QAE@PAV?$basic_streambuf@DU?$char_traits@D@std@@@1@@Z

?flush@?$basic_ostream@DU?$char_traits@D@std@@@std@@QAEAAV12@XZ
MSVC     public: class std::basic_ostream<char,struct std::char_traits<char> > & __thiscall std::basic_ostream<char,struct std::char_traits<char> >::flush(void)
LLVM     public: class std::basic_ostream<char, struct std::char_traits<char>> & __thiscall std::basic_ostream<char, struct std::char_traits<char>>::flush(void)
BNJA NEW class std::basic_ostream<char,struct std::char_traits<char> >& __thiscall std::basic_ostream<char,struct std::char_traits<char> >::flush(std::basic_ostream<char,struct std::char_traits<char> >* this)
BNJA OLD ?flush@?$basic_ostream@DU?$char_traits@D@std@@@std@@QAEAAV12@XZ

?max_size@?$allocator_traits@V?$allocator@G@std@@@std@@SA_KAEBV?$allocator@G@2@@Z
MSVC     public: static unsigned __int64 __cdecl std::allocator_traits<class std::allocator<unsigned short> >::max_size(class std::allocator<unsigned short> const & __ptr64)
LLVM     public: static unsigned __int64 __cdecl std::allocator_traits<class std::allocator<unsigned short>>::max_size(class std::allocator<unsigned short> const &)
BNJA NEW uint64_t __cdecl std::allocator_traits<class std::allocator<uint16_t> >::max_size(class std::allocator<uint16_t> const& __ptr64)
BNJA OLD ?max_size@?$allocator_traits@V?$allocator@G@std@@@std@@SA_KAEBV?$allocator@G@2@@Z

Couple slight deviations from MSVC/LLVM but I think they are just thisptr insertion and lack of type system support for public/protected/static status.

I really appreciate seeing movement in this area @CouleeApps 😃

Now included, even if nobody here asked for it: bare names of the construction .?AV<name> or .?AU<name>. I’m not sure what the difference between AV and AU is, so they’re just handled the same. Specifically you should get something like:

>>> demangle_ms(Architecture['x86'], '.?AVMyType@@')
(<type: immutable:VoidTypeClass 'void'>, ['MyType'])

I’ve only seen these used in RTTI TypeDescriptor name fields, so if you’re working on an RTTI plugin this may be of use 😃