Anthos 1.2 Time Sync issues on GKE On-Prem nodes

When you login into your GKE-OP nodes you might find out the time is synced with your ESXi host rather then with the Timeserver configured in you DHCP Options or Static IP files used for GKE-OP clusters provisioning.

This issue is actually related to Ubuntu 18 and is connected with settings of timesyncd service.

To see if you are experiencing the issue run

sudo SYSTEMD_LOG_LEVEL=debug /lib/systemd/systemd-timesyncd

ubuntu@gke-03-user0103:~$ sudo SYSTEMD_LOG_LEVEL=debug /lib/systemd/systemd-timesyncd
Failed to create state directory: Permission denied
ubuntu@gke-03-user0103:~$ sudo SYSTEMD_LOG_LEVEL=debug /lib/systemd/systemd-timesyncd
Added new server
Added new server
Selected server
Resolved address for
Selected address of server
Connecting to time server (
Sent NTP request to (
Server has too large root distance. Disconnecting.
Waiting after exhausting servers.

Root cause: there is network delay that causes timeout for the response from NTP

Solution: There is no permanent solution for this issue as the settings of NTP are created when Nodes are deployed using the DHCP or StaticIP files. You can only fix this issue after your nodes are deployed. The settings will be lost when you redeploy.

To workaround this issue set edit the timesyncd.conf file and set RootDistanceMaxSec=20 (you might need to find our the honey spot)

sudo cat /etc/systemd/timesyncd.conf






Now you should check if connection works fine

ubuntu@gke-03-user0103:~$ sudo SYSTEMD_LOG_LEVEL=debug /lib/systemd/systemd-timesyncd
Added new server

freq offset : +0 (0 ppm)
interval/delta/delay/jitter/drift 64s/+0.033s/0.001s/0.000s/+0ppm
Synchronized to time server (


Wprowadź swoje dane lub kliknij jedną z tych ikon, aby się zalogować:


Komentujesz korzystając z konta Wyloguj /  Zmień )

Zdjęcie z Twittera

Komentujesz korzystając z konta Twitter. Wyloguj /  Zmień )

Zdjęcie na Facebooku

Komentujesz korzystając z konta Facebook. Wyloguj /  Zmień )

Połączenie z %s