PX4-Autopilot: [Bug] Time is not tracked correctly after power cycle or reboot

Describe the bug

I observed that on some pixhawks that date and time is not tracked correctly over reboots or power cycle. This can be observed using the date or system_time get command before and after a reboot. This issue does not occur on all pixhawks. I observed this issue already on multiple holybro pix32v5 and also on holybro pixhawk 6x.

Around 30% of the pix32v5 seem to suffer from it and in the worst case the time jumps back significantly. I just recently did a few extensive tests with one pix32v5 where I observed following:

real time: 6min -> after a reboot only 4 min elapsed
real time: 10min -> after a reboot only 3 min elapsed
real time: 48min -> after a reboot only 14 min elapsed

It looks like the time is tracked around 3 times slower than it should, although for smaller duration the factor seems smaller.

My current guess is that it is either a hardware issue (i.e. the LSE oscillator is ticking way slower than it should) or it is a misconfiguration of the RTC or LSE that leads to inconsistent behavior across pixhawks.

Note, that while the pixhawk is running the time is tracked correctly, only after a reboot or power cycle the time jumps back.

To Reproduce

To reproduce you just need a pixhawk and an interface to the nsh console. But your pixhawk might be fine thus it is hardware dependent.

Power on
Wait a few minutes and execute date into the console
Execute reboot
Again execute date and compare the time you extracted in step 2.
If you have a faulty pixhawk you will observe a significant time jump.

Expected behavior

The time should be tracked correctly over reboots, power cycle or longer power off durations.

Screenshot / Media

No response

Flight Log

Software Version

Release 1.13.3

Flight controller

Holybro: pix32v5, pixhawk 6x

Vehicle type

None

How are the different components wired up (including port information)

No response

Additional context

No response

About this issue

Original URL
State: open
Created 10 months ago
Comments: 24 (15 by maintainers)

Most upvoted comments

The RTC’ time is used at boot to set the NuttX time. NuttX time. marches along off the system tick interrupt.

System without a RTC Xtal use the HSE Highspeed external oscillator. Time set operations set the RTC. (from GPS, QGC, command line etc) When power is removed the RTC stops counting and retains the last set time if the battery is good…

System With a RTC Xtal use the LSE Lowspeed external oscillator. Time set operations set the RTC. (from GPS, QGC, command line etc) When power is removed the RTC keeps counting from the last set time if the battery is good.

davids5 on Sep 8, 2023

There are 2 element being tested. The crystal and the battery. This is why reset is not used.

Start a stop watch and unplug the FMU Wait 3 min or so power on the FMU and issue the date command: date

If the date is reset to Jan 1, then the battery is bad or dead.

If the xtal is not running then the date will be the last set value+time to boot.

davids5 on Sep 8, 2023

We tested it on two Pix32 v5 and one Pixhawk 5X, and the unable to reproduce the issue. We will test a few more board to see if we can find one that can reproduce.

@vincentpoont2 - the test can not be made with QGC connected. Please use the debug console

davids5 on Sep 8, 2023

@dmammolo Are the FC connected to a GPS or network and not connected to QGC via USB?

The test I normally do is with no GPS, no QGC, and no network connection boot the system. Issue the date command: date -s "Sep 13 10:00:00 2020" Start a stop watch and unplug the FMU Wait 3 min or so power on the FMU and issue the date command: date I verify the time is with in a second or 2.

@dmammolo Do you have access to a frequency meter or an oscilloscope? Can you measure the 32Khz xtal?

@vincentpoont2 Can you please verify this.

davids5 on Sep 7, 2023