Network Time Protocol (NTP) Explained: Synchronizing Internet Clocks

This section covers NTS architecture, software recommendations, and deployment best practices. NTS (Network Time Security) was developed starting in March 2015 and standardized as RFC 8915 after approximately 5 years of development. NTS consists of two components: NTS Key Establishment (NTS-K) for TLS-based authentication on TCP port 4460, and NTS Extension Fields (NTS-E) for cryptographic message authentication. NTS provides server authentication using X.509 certificates, replay attack prevention through cookie-based state tracking, and privacy protection by rotating cookies to prevent correlation of queries across network locations. Recommended NTP software includes ntpsec (modern implementation with NTS support, actively maintained), chrony (standard on Red Hat Linux, good performance), and ntpd (reference implementation, large codebase, slower security response). Not recommended: network devices as NTP servers (use clients instead). Best practices include using modern software, deploying internal servers for larger networks, configuring at least 3 servers for redundancy and attack detection, and using NTS for internal servers to prevent amplification attacks. Configuration involves installing via package manager, configuring servers in configuration files, using NTS servers with proper authentication, and configuring firewall rules for UDP 123 and TCP 4460. Server configuration requires certificate setup, firewall rules, and access control.

The Network Time Protocol (NTP) was originally designed without any security features, making it vulnerable to attacks. Attackers exploited this by using NTP in amplification attacks during Distributed Denial of Service (DDoS) operations. Approximately thirty years after its introduction, security features were added through NTPsec (Secure Network Time Protocol), which implements additional protections and removes existing vulnerabilities from the original codebase.

Network Time Protocol (NTP) is one of the oldest Internet protocols, operating since before 1985 for clock synchronization. Early versions (NTPv1, NTPv2) lacked authentication. NTPv3 added pre-shared key authentication but required out-of-band key distribution due to eavesdropping concerns. NIST distributes keys via postal mail to prevent network interception. However, establishing keys with billions of devices is impractical. NTPv4 introduced Autokey with 32-bit seeds that can be brute-forced. Network Time Security (NTS) is a newer proposal. The fundamental challenge is that NTP servers themselves need accurate time, creating circular dependencies. This historical context shows how security protocols evolve but often retain fundamental trust assumptions that create vulnerabilities.

Traditional NTP (Network Time Protocol) used by default in Linux distributions is insecure as it is unauthenticated and unencrypted, typically using only one pool server. A more secure alternative uses NTS (Network Time Security), which provides encryption and authentication. Using multiple time pools (at least four) with NTS ensures that at least two sources must agree on a time update before it is accepted, preventing malicious pool servers from forcing incorrect time updates.

Enterprise security is built on the CIA triad: Confidentiality (authorized data access), Integrity (data protection from tampering), and Availability (system accessibility). Key concepts include vulnerability (system weakness), exploit (attack tool), threat (exploitation potential), and mitigation (protection measures). Network attacks include DoS attacks (TCP SYN flood exploits handshake), spoofing attacks (fake IP/MAC addresses), reflection/amplification attacks (exploiting DNS/NTP servers), and man-in-the-middle attacks (ARP spoofing). Reconnaissance attacks gather target information through NSLOOKUP and WHOIS queries. Malware includes viruses, worms, and trojan horses. Social engineering targets human vulnerabilities through phishing, spear phishing, vishing, and tailgating.

RHEL/CentOS 7 supports two NTP client implementations: Chrony (default) and traditional NTP (ntpd). Chrony is recommended for systems with intermittent network connectivity, congested networks, or frequent reboots due to its efficient time synchronization and support for the iburst option. Traditional NTP (ntpd) is better suited for production servers that remain permanently powered on. To configure Chrony, enable the service with 'systemctl enable chronyd', start it with 'systemctl start chronyd', and verify synchronization using 'chronyc sources' or 'chronyc tracking'. To configure NTP, install the package with 'yum install ntp', enable the service with 'systemctl enable ntpd', and verify with 'ntpq -p'.

Time synchronization is critical in Linux systems, especially for distributed environments where time drift causes issues in logs, troubleshooting, and security. NTP (Network Time Protocol) synchronizes time between computers over networks. In openSUSE Leap, Chrony is the default NTP implementation - fast, accurate, and effective on systems with intermittent connections. Chrony consists of chronyd (daemon) and chronyc (command-line tool). Configuration options include start timing (manual, periodic, or on boot), source type (dynamic for DHCP or static for predefined servers), and time server management. Manual configuration via /etc/chrony.conf uses 'pool' directives for reliable sources like pool.ntp.org. Key directives include iburst (faster initial sync), pull (sync frequency using base-2 algorithm), minpoll/maxpoll (interval ranges), and local stratum (clock reliability). Chronyc provides runtime monitoring with detailed metrics: Reference ID, Stratum, Last Update, System Time, Offset, RMS Offset, Frequency, Residual Frequency, Skew, Root Delay, Root Dispersion, Update Interval, and Leap Status. For offline mode during boot, add 'offline' to server lines in /etc/chrony.conf. Changes via Chronyc are not permanent - edit /etc/chrony.conf for persistent changes.

Chrony is the enhanced time synchronization service introduced in RHEL7/CentOS7 that replaces the traditional NTP daemon (ntpd), offering superior performance with intermittent network connectivity, faster synchronization speeds, and better handling of sudden clock rate changes due to temperature variations; it uses the 'timedatectl' command for configuration and status checks, with configuration files located in /etc/chrony, and requires enabling the chronyd service using systemctl.

To configure an NTP server on CentOS 7, first remove chrony using 'yum remove chrony', then install NTP with 'yum install ntp', enable the service with 'systemctl enable ntpd', and verify the configuration in /etc/ntp.conf which uses default CentOS server pools; the server can be configured to sync with additional NTP peers by adding their IP addresses to the configuration file.

Red Hat 8 and newer versions use Chrony instead of the traditional NTP daemon (ntpd) for time synchronization. Chrony is a more efficient and accurate time synchronization daemon that works well even with low network bandwidth. To configure Chrony, the chrony service must be started and enabled using 'systemctl start chronyd' and 'systemctl enable chronyd'. The configuration file is located at /etc/chrony.conf, where NTP servers can be specified. The 'chronyc sources' command shows which time servers the system is currently synchronizing with, including the offset in microseconds.

This lecture covers three major distributed database systems: CockroachDB (an open-source distributed transactional database using Raft consensus and hybrid logical clocks for MVCC), Google Spanner (a geo-replicated database achieving external consistency through TrueTime hardware synchronization with atomic clocks and GPS), and MongoDB (a document-oriented database with JSON-based storage, auto-sharding, and Server-Side Public License). The key architectural differences include CockroachDB's shared-nothing design with RocksDB storage, Spanner's shared-disk architecture with physical denormalization, and MongoDB's centralized routing with master-slave replication.

CockroachDB implements lock-free transactions using a two-phase commit variation without traditional locking mechanisms. When a transaction starts, it writes itself to a central system transaction table and writes tentative values (intents) to the key-value store with a special flag indicating they belong to an active transaction. Commit involves updating the transaction table from 'pending' to 'committed' and asynchronously upgrading intents to final values. This design is lightweight because the client receives confirmation immediately after commit, while cleanup happens in the background. The transaction table serves as the authoritative source of truth for transaction status. CockroachDB uses hybrid logical clocks (HLC) instead of Google's GPS/atomic clock approach, combining wall times with logical components to track causality in distributed systems.

CockroachDB had two bugs: (1) Timestamp cache allowed two transactions with same timestamp to use each other's entries, causing double inserts on the same read set; (2) Single-statement transactions could be retried with new timestamps, causing the same insert to commit twice. Solutions: clear transaction ID when timestamp collisions occur, and distinguish retriable from non-retriable errors.

CockroachDB was inspired by Google Spanner, which uses a similar distributed architecture. Google Spanner is used internally for virtually all Google products, making it one of the most widely used databases in the world. CockroachDB provides a similar architecture but is designed to be accessible as a public cloud database service. The key insight is that building distributed systems requires different mental models than traditional single-node databases, particularly around range management and sequence allocation.

CockroachDB is an open source implementation of Spanner's architecture. However, it lacks true time and instead uses NTP (Network Time Protocol) for clock synchronization, which can drift by 200 milliseconds. The speaker explains that this drift means CockroachDB always incurs a latency cost for reads, approximately 200 milliseconds per read, which is too slow for many use cases. Despite this limitation, CockroachDB passes Jepsen tests and is a viable solution for those who can tolerate the latency cost.