For the future, the just released systemd 250 integrates a similar method for maintaining a rough clock for RTC-less systems into systemd-timesyncd:
A new setting SaveIntervalSec= has been added to systemd-timesyncd, which may be used to automatically save the current system time to disk in regular intervals. This is useful to maintain a roughly monotonic clock even without RTC hardware and with some robustness against abnormal system shutdown.
I’ve added a note to the post linking here. I read the 250 release notes yesterday, which probably got the thought in my mind today, but then I totally forgot about this change. It’s even literally directly below the change I was looking at, lock-valid-range-usec-max. So frustrating I missed. :)
As I mention in the note, I don’t use timesyncd. Though if I did, it runs much later in the boot process than the clock-epoch, simultaneously with other services. I’d prefer having the clock reasonably setup before then.
systemd compares the system time to a builtin epoch, usually the release or build date of systemd. If it finds the system time is before this epoch, it resets the clock to the epoch
I think this is a great example of why systemd is capable of so many features. You could totally let this be done by a different service, but then you’ll have to re-introduce this kind of trigger into systemd - in a way that it does run before all the things that need DNS. Otherwise your init system plays “crash everything” on bootup which is totally worthless.
This can easily run as your own unit though. There is no need for systemd to do it specifically. You can create a job like that and make sure it runs before network.target.
While true, you cannot run a unit as early in the boot process as this systemd method, which happens in main before targets and units are started. In most cases this doesn’t matter, but it’s nice to not have all my system logs starting 41 years ago.
I see there point, however I prefer “obviously wrong time” to “maybe from the last boot or maybe the bootup was actually hanging for minutes” timestamps.
as you see in the article, that comes at some drawbacks and might cause manual intervention when a somewhat wrong timestamp could be good enough for everything to come back up where NTP will be used to correct the timestamps.
I guess it’s something where you have to decide for each machine whether audibility of timestamps or resilience against manual intervention is more important.
It seems to me that what you really need, then, is a log message saying “time changed from x to y”? Does systemd’s implementation include such a message?
Timer files do not need a unit defined. By default, they run the service unit that matches the same name, ignoring “.timer”. Yes, I’m not sure where my copy and paste of the WantedBy went.
As for the commands, yes typos. I’ll fix. (edit, fixed! thanks)
That’s activating the timer to begin updating the file’s modification time, which was an arbitrary value after bootup. You can set it lower if you want, and if your reboot took longer than 17 minutes, it well run sooner anyways. The clock is set from the mtime well before that.
If you’re trying to attack DoH by that you’re not making a good argument. If you resolve that DNS without https you still might want to verify certificates. Or you’re even at the point that you resolved your IP address. Fine and now what ? Talk https to the service and fail to verify their certificates ? Or pray that timesyncd was running in between so it may randomly work.
Generally speaking I do not like CA based encrypted communication. It breaks down easily and there are many ways to break it. You should be able to get the current time without much hassle especially without DoH or DoT getting involved which can be broken without proper time.
Fully agreed that the CA system has many (well-documented) flaws. But it seems to me that the only realistic alternative at this point is plaintext, which seems worse?
Ha, usually I’m the guy complaining folks didn’t RTFA. I certainly did, and I reckon you missed the section about systemd-timesyncd that was added. Or maybe I was responding to other comments in this discussion about systemd-timesyncd.
You are correct about it being core systemd behavior. But only the read part. The fact I have to manually create a new unit file myself to periodically touch the timestamp file makes this pretty awkward. Also as discussed 250 has a different mechanism in systemd-timesyncd which applies later, maybe too late to be useful. It’s all a bit of a mess.
Crony doesn’t really solve the very-early-in-bootstrap problem though, it’s only going to set the clock once it’s started. It’d be nice if crony touched the epoch file systemd is expecting. Although /usr/lib is a weird place to store dynamic state like this. Is /usr/lib/unix-epoch used for anything else? My brief searches suggest it’s not.
Although /usr/lib is a weird place to store dynamic state like this.
This awkwardness is because it’s intended for people making system images (like the SD card images for a Pi) to set the epoch without needing to rebuild systemd to set the built-in epoch. That it works here is a nice side effect.
It’d be nice if crony touched the epoch file systemd is expecting.
Since all systemd cares about is the mtime of the file, you might be able to adjust paths in chrony.conf to it. I’m not super familiar with chronyd, but maybe rtcfile would work? It would be great if it had a directive specifically for setting the mtime (or had ability to run a command after sync).
Yep, post only talks about NTP, and my edit in response to an earlier comment even specifically says I don’t use timesyncd.
Chronyd, and other NTP-like tools, tend to start very late in the boot process, as they want networking. By having core systemd set it so early in boot, before units run, the time delta they observe once NTP is running is much smaller. (No more text saying my system booted “41 years ago” for example text)
I wouldn’t get good GPS reception, don’t have a great way to run cabling for an antenna, and if I’m going through that expense, I might as well spend the few dollars on a RTC hat, instead of a software solution.
An alternative approach could be to use ips in your initial ntp setup. There’s a few endpoints which haven’t moved in years and hardcoding a few of those addresses could allow you to get real time before using DoH.
Great trick, I’ll adding it to all my RPis!
For the future, the just released systemd 250 integrates a similar method for maintaining a rough clock for RTC-less systems into
systemd-timesyncd:(From https://lwn.net/Articles/879739/)
I’ve added a note to the post linking here. I read the 250 release notes yesterday, which probably got the thought in my mind today, but then I totally forgot about this change. It’s even literally directly below the change I was looking at,
lock-valid-range-usec-max. So frustrating I missed. :)As I mention in the note, I don’t use timesyncd. Though if I did, it runs much later in the boot process than the clock-epoch, simultaneously with other services. I’d prefer having the clock reasonably setup before then.
I think this is a great example of why systemd is capable of so many features. You could totally let this be done by a different service, but then you’ll have to re-introduce this kind of trigger into systemd - in a way that it does run before all the things that need DNS. Otherwise your init system plays “crash everything” on bootup which is totally worthless.
This can easily run as your own unit though. There is no need for systemd to do it specifically. You can create a job like that and make sure it runs before network.target.
While true, you cannot run a unit as early in the boot process as this systemd method, which happens in
mainbefore targets and units are started. In most cases this doesn’t matter, but it’s nice to not have all my system logs starting 41 years ago.I see there point, however I prefer “obviously wrong time” to “maybe from the last boot or maybe the bootup was actually hanging for minutes” timestamps.
as you see in the article, that comes at some drawbacks and might cause manual intervention when a somewhat wrong timestamp could be good enough for everything to come back up where NTP will be used to correct the timestamps.
I guess it’s something where you have to decide for each machine whether audibility of timestamps or resilience against manual intervention is more important.
It seems to me that what you really need, then, is a log message saying “time changed from x to y”? Does systemd’s implementation include such a message?
AFAICT, journald does not, but the NTP client I use does log when it adjusts time when it starts up, which is good enough.
As for “maybe from the last boot” I usually invoke
journalctl --boot=0unless I’m specifically looking for logs from previous boots.Ah, by “systemd’s implementation” I meant systemd-timesyncd. Though the journal noticing this would probably work too, now that I think about it
Doesn’t the timer file need a service point?
And shouldn’t the “enable and start” command be
systemctl …rather thansystemd?And shouldn’t there be a
WantedBy=timers.target?Timer files do not need a unit defined. By default, they run the service unit that matches the same name, ignoring “.timer”. Yes, I’m not sure where my copy and paste of the
WantedBywent.As for the commands, yes typos. I’ll fix. (edit, fixed! thanks)
Also, I don’t understand why it should wait 5 minutes before activation?
That’s activating the timer to begin updating the file’s modification time, which was an arbitrary value after bootup. You can set it lower if you want, and if your reboot took longer than 17 minutes, it well run sooner anyways. The clock is set from the mtime well before that.
Thanks
I’ve added a note to the post mentioning this was an arbitrary decision.
I have been using
fake-hwclock+havegedon my managed embedded devices (Olinuxino Lime2).DNS over TLS was a great idea
If you’re trying to attack DoH by that you’re not making a good argument. If you resolve that DNS without https you still might want to verify certificates. Or you’re even at the point that you resolved your IP address. Fine and now what ? Talk https to the service and fail to verify their certificates ? Or pray that timesyncd was running in between so it may randomly work.
Generally speaking I do not like CA based encrypted communication. It breaks down easily and there are many ways to break it. You should be able to get the current time without much hassle especially without DoH or DoT getting involved which can be broken without proper time.
Fully agreed that the CA system has many (well-documented) flaws. But it seems to me that the only realistic alternative at this point is plaintext, which seems worse?
If you care about accurate time don’t use systemd-timesyncd. It’s really not good. Chrony is the best choice for timekeeping on many applications.
Chrony’s solution for “there’s no RTC to set the clock” is to use the last timestamp on the drift file it writes during regular usage. https://chrony.tuxfamily.org/faq.html#_what_if_my_computer_does_not_have_an_rtc_or_backup_battery
I reckon you didn’t read the article before writing that comment. Why? Because it doesn’t talk about systemd-timesyncd (this mechanism is core systemd behaviour: https://github.com/systemd/systemd/blob/b049b48c4b6e60c3cbec9d2884f90fd4e7013219/src/core/main.c#L1653), and because it does describe the same policy of creating and periodically updating an epoch file.
Ha, usually I’m the guy complaining folks didn’t RTFA. I certainly did, and I reckon you missed the section about systemd-timesyncd that was added. Or maybe I was responding to other comments in this discussion about systemd-timesyncd.
You are correct about it being core systemd behavior. But only the read part. The fact I have to manually create a new unit file myself to periodically touch the timestamp file makes this pretty awkward. Also as discussed 250 has a different mechanism in systemd-timesyncd which applies later, maybe too late to be useful. It’s all a bit of a mess.
Crony doesn’t really solve the very-early-in-bootstrap problem though, it’s only going to set the clock once it’s started. It’d be nice if crony touched the epoch file systemd is expecting. Although /usr/lib is a weird place to store dynamic state like this. Is /usr/lib/unix-epoch used for anything else? My brief searches suggest it’s not.
This awkwardness is because it’s intended for people making system images (like the SD card images for a Pi) to set the epoch without needing to rebuild systemd to set the built-in epoch. That it works here is a nice side effect.
Since all systemd cares about is the mtime of the file, you might be able to adjust paths in
chrony.confto it. I’m not super familiar with chronyd, but maybertcfilewould work? It would be great if it had a directive specifically for setting the mtime (or had ability to run a command after sync).Yep, post only talks about NTP, and my edit in response to an earlier comment even specifically says I don’t use timesyncd.
Chronyd, and other NTP-like tools, tend to start very late in the boot process, as they want networking. By having core systemd set it so early in boot, before units run, the time delta they observe once NTP is running is much smaller. (No more text saying my system booted “41 years ago” for example text)
The author added a comment from here about systemd-timesyncd about 7 hours ago.
This kind of situation is why I have a local stratum 1 NTP server.
I wouldn’t get good GPS reception, don’t have a great way to run cabling for an antenna, and if I’m going through that expense, I might as well spend the few dollars on a RTC hat, instead of a software solution.
An alternative approach could be to use ips in your initial ntp setup. There’s a few endpoints which haven’t moved in years and hardcoding a few of those addresses could allow you to get real time before using DoH.