salt: [BUG] metadata grains failing since 3006.3

Description metadata grain is not returning successfully since 3006.3. This did work in 3006.2.

Setup EC2 host in AWS metadata_server_grains: True in minion config

Steps to Reproduce the behavior

$ sudo salt-call grains.get dynamic:instance-identity --out=json
[CRITICAL] Failed to load grains defined in grain file metadata.metadata in function <LoadedFunc name='metadata.metadata'>, error:
Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/__init__.py", line 1150, in grains
    ret = funcs[key](**kwargs)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 159, in __call__
    ret = self.loader.run(run_func, *args, **kwargs)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1245, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/loader/lazy.py", line 1260, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/grains/metadata.py", line 87, in metadata
    return _search()
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/grains/metadata.py", line 64, in _search
    ret[line] = _search(prefix=os.path.join(prefix, line + "/"))
  File "/opt/saltstack/salt/lib/python3.10/site-packages/salt/grains/metadata.py", line 53, in _search
    linedata["headers"].get("Content-Type", "text/plain")
KeyError: 'headers'
{
    "local": ""
}

Expected behavior command should return some valid json, along the lines of:

{
  "accountId" : "xxx",
  "architecture" : "x86_64",
  "availabilityZone" : "us-east-1a",
  "billingProducts" : null,
  "devpayProductCodes" : null,
  "marketplaceProductCodes" : null,
  "imageId" : "ami-xxx",
  "instanceId" : "i-xxx",
  "instanceType" : "t2.large",
  "kernelId" : null,
  "pendingTime" : "2023-05-02T13:54:39Z",
  "privateIp" : "xxx.xxx.xxx.xxx",
  "ramdiskId" : null,
  "region" : "us-east-1",
  "version" : "2017-09-30"
}

Curl commands against the metadata API return expected results. curl http://169.254.169.254/latest/dynamic/instance-identity/document/

Versions Report

Salt Version:
          Salt: 3006.3

Python Version:
        Python: 3.10.13 (main, Sep  6 2023, 02:11:27) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.14.6
      cherrypy: Not Installed
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.2
       libgit2: Not Installed
  looseversion: 1.0.2
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 22.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.9.8
        pygit2: Not Installed
  python-gnupg: 0.4.8
        PyYAML: 6.0.1
         PyZMQ: 23.2.0
        relenv: 0.13.10
         smmap: Not Installed
       timelib: 0.2.4
       Tornado: 4.5.3
           ZMQ: 4.3.4

System Versions:
          dist: ubuntu 22.04.3 jammy
        locale: utf-8
       machine: x86_64
       release: 6.2.0-1009-aws
        system: Linux
       version: Ubuntu 22.04.3 jammy