openpilot: Ford Q3 Hybrids have occasional incorrect PCM message checksums

This is causing controls mismatches for some users, including at least a Maverick and Escape. Haven’t hit it on our Bronco yet however. I checked all three message checksums we check in panda (one from RCM, one from ABS, and one from PCM) and only the PCM message has these mismatches (it’s behind a gateway, other two are on the same bus…).

Results: 2114
f8eaaccd2a90aef8|2023-05-05--12-27-53--18  FORD ESCAPE 4TH GEN    addr 0x202: 1 mismatches / 3000 msgs
f8eaaccd2a90aef8|2023-05-05--12-58-21--10  FORD ESCAPE 4TH GEN    addr 0x202: 1 mismatches / 2999 msgs
bd37e43731e5964b|2023-05-05--21-42-22--19  FORD MAVERICK 1ST GEN  addr 0x202: 3 mismatches / 3000 msgs
f8eaaccd2a90aef8|2023-05-05--12-27-53--22  FORD ESCAPE 4TH GEN    addr 0x202: 3 mismatches / 3000 msgs
933ad9d04ac82693|2023-02-08--09-19-55--22  FORD MAVERICK 1ST GEN  addr 0x202: 1 mismatches / 3000 msgs
f8eaaccd2a90aef8|2023-05-05--12-58-21--192 FORD ESCAPE 4TH GEN    addr 0x202: 2 mismatches / 2999 msgs
bd37e43731e5964b|2023-05-05--21-42-22--7   FORD MAVERICK 1ST GEN  addr 0x202: 1 mismatches / 3000 msgs
933ad9d04ac82693|2023-03-23--12-18-36--15  FORD MAVERICK 1ST GEN  addr 0x202: 1 mismatches / 2999 msgs
bd37e43731e5964b|2023-05-04--17-29-57--31  FORD MAVERICK 1ST GEN  addr 0x202: 1 mismatches / 3000 msgs
933ad9d04ac82693|2023-03-07--15-35-14--16  FORD MAVERICK 1ST GEN  addr 0x202: 1 mismatches / 3000 msgs
bd37e43731e5964b|2023-05-05--21-42-22--20  FORD MAVERICK 1ST GEN  addr 0x202: 1 mismatches / 3000 msgs
bd37e43731e5964b|2023-05-05--16-59-49--26  FORD MAVERICK 1ST GEN  addr 0x202: 1 mismatches / 3000 msgs
bd37e43731e5964b|2023-05-04--20-59-54--8   FORD MAVERICK 1ST GEN  addr 0x202: 1 mismatches / 3000 msgs
bd37e43731e5964b|2023-05-04--17-29-57--18  FORD MAVERICK 1ST GEN  addr 0x202: 2 mismatches / 3000 msgs
bd37e43731e5964b|2023-05-04--12-06-21--5   FORD MAVERICK 1ST GEN  addr 0x202: 2 mismatches / 3000 msgs
bd37e43731e5964b|2023-05-05--21-42-22--27  FORD MAVERICK 1ST GEN  addr 0x202: 1 mismatches / 2999 msgs

Matches by addr {514: 6199603, 1045: 6211839, 145: 12395638}
Mismatches by addr {514: 23793, 1045: 0, 145: 0}

Checked segments:
FORD BRONCO SPORT 1ST GEN 793
FORD FOCUS 4TH GEN 147
FORD ESCAPE 4TH GEN 243
FORD EXPLORER 6TH GEN 205
FORD MAVERICK 1ST GEN 726

Checked dongles:
FORD BRONCO SPORT 1ST GEN 4
FORD FOCUS 4TH GEN 1
FORD ESCAPE 4TH GEN 3
FORD EXPLORER 6TH GEN 4
FORD MAVERICK 1ST GEN 3

Checked platforms:
FORD BRONCO SPORT 1ST GEN {'54827bf84c38b14f': 717, '62241b0c7fea4589': 38, 'bbe75f19590890c4': 33, '5a08e8ef78cebd9f': 5}
FORD FOCUS 4TH GEN {'86d00e12925f4df7': 147}
FORD ESCAPE 4TH GEN {'f8eaaccd2a90aef8': 205, '26b2cace68e36212': 34, '62241b0c7fea4589': 4}
FORD EXPLORER 6TH GEN {'59a107f5793d9cc0': 22, '62241b0c7fea4589': 16, 'f0709d2bc6ca451f': 151, '8e1bab39e558e773': 16}
FORD MAVERICK 1ST GEN {'bd37e43731e5964b': 374, 'c845bd8c366e6f3d': 96, '933ad9d04ac82693': 256}

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 32 (12 by maintainers)

Most upvoted comments

I did a lot of work around this in the comma discord. Adeeb banned me, so forgive me if I remember incorrectly. It should be under Ford then a thread for “Ford hybrid broke”.

I took the messages that Koopa got behind the gateway for the different BUSes and then ran through all the messages on Identifier 514 (hex 0x202) for checksum issues:

image

Anyways most if not all the data is in that thread.

The bounty is for root causing the incorrect checksums and getting the Hybrids shipped.

on

2023 Ford Maverick Hybrid

This is good news for you and could solve the issue for others. Can you get a drive to do public share of the route?

Matt the gas for a few seconds is 100% reproduce or ACC acceleration from 60 to 80mph is like 75% reproduce. I can get you some more logs. If you look on the discord at #ford hybrid broke there is the story

Mash the gas? Tap the gas? Sorry

100% full throttle for a few seconds will rack up enough mismatches to throw an error. Also, it is a safety concern that code is blocked while mismatches count up. You get a good half a second or so of comma holding course regardless of whatever is changing on the road.

I’m happy to collect more can data if needed. Anything the community can work on? The acceptable solution space is unclear.

Is this going to be in before 1.0? I can get whatever data is needed to help you solve it. -koopaloopz