json: String type change breaks C++ type matching

The workaround I mentioned before is still the only way to solve this I could think of so far. Any other solutions I considered are breaking changes and would have to wait until 4.0.

The caveat is, that even my workaround requires a minor modification of the library and I haven’t done any significant testing to ensure it doesn’t break anything else.

Here’s the code to disable conversions between basic_json types with mismatched string types: (alt_string is the alternative string type from the library unit test, it’s a thin wrapper around std::string. Replace that with your string type.)

namespace nlohmann {
template<>
struct adl_serializer<alt_string> {
template<typename BasicJsonType, std::enable_if_t<std::is_same_v<typename BasicJsonType::string_t, alt_string>, int> = 0>
static void from_json(BasicJsonType&& j, alt_string& s) {
    assert(j.is_string());
    s = j.template get_ref<const alt_string &>();
}

template<typename BasicJsonType, std::enable_if_t<std::is_same_v<typename BasicJsonType::string_t, alt_string>, int> = 0>
static void to_json(BasicJsonType& j, const alt_string& s) {
    ::nlohmann::to_json(j, s);
}
};

template<>
struct adl_serializer<std::string> {
template<typename BasicJsonType, std::enable_if_t<std::is_same_v<typename BasicJsonType::string_t, std::string>, int> = 0>
static void from_json(BasicJsonType&& j, std::string& s) {
    assert(j.is_string());
    s = j.template get_ref<const std::string &>();
}

template<typename BasicJsonType, std::enable_if_t<std::is_same_v<typename BasicJsonType::string_t, std::string>, int> = 0>
static void to_json(BasicJsonType& j, const std::string& s) {
    ::nlohmann::to_json(j, s);
}
};
}  // namespace nlohmann

This works by SFINAE-disabling the from_json/to_json functions for mismatched string types.

In addition, the following change to include/nlohmann/json.hpp is required for the above code to compile.

1. Find this member function of basic_json:

   /// @brief create a JSON value from compatible types
   /// @sa https://json.nlohmann.me/api/basic_json/basic_json/
   template < typename CompatibleType,
              typename U = detail::uncvref_t<CompatibleType>,
              detail::enable_if_t <
                  !detail::is_basic_json<U>::value && detail::is_compatible_type<basic_json_t, U>::value, int > = 0 >
   basic_json(CompatibleType && val) noexcept(noexcept( // NOLINT(bugprone-forwarding-reference-overload,bugprone-exception-escape)
               JSONSerializer<U>::to_json(std::declval<basic_json_t&>(),
                                          std::forward<CompatibleType>(val))))
   {
       JSONSerializer<U>::to_json(*this, std::forward<CompatibleType>(val));
       set_parents();
       assert_invariant();
   }
   

2. Replace the template parameter U in detail::is_compatible_type<basic_json_t, U>::value with CompatibleType.

Again, I haven’t done any further testing. Once I confirm that this modification doesn’t break anything else, I’ll submit a PR.

@falbrechtskirchinger my custom string class is something like @agauniyal’s secure_string. I used std::wstring only as an example, to avoid pasting code of my string class here. It is something like below. With this approach I can securely allocate and clear memory used by std::string object too:

class secure_string
{
private:
  std::string<char, std::char_traits<char>, secure_allocator>* str;
public:
  // public interface which mimics std::string and (mostly) is a thin wrapper around `str`, e.g.
  size_t size() const { return str->size(); }
};

I’ll try if can get you a mve for secure_allocator. Your understanding about it’s secure nature is correct as well, I just want to add that it disables SSO by manually allocating bigger strings even in the case of small ones, such that nothing is stored on the stack.

This might be related to #3425. I’ll check if any of the provided examples trigger the assertions I’ve added to at least catch some problematic conversions between basic_json types.

And as an aside, std::wstring as StringType is not well supported. There’re too many places where it’s assumed that the underlying char type is 8bit wide. Parsing std::wstring on the other hand is well defined.

I am aware of the issue, but have not found the time to look into it.