Most modern network adapters feature a PTP hardware clock (PHC). The PHC is usually implemented as a free running clock, which starts from 0 at boot time, and progresses at the core frequency of the network adapter. As each timestamp needs to be translated by the kernel software, the need for many timestamps can have a huge impact on the data path.
As can be seen in Figure 1 - Timestamp conversion in Software, this flow also introduces additional errors in the timestamp conversion in the driver, as well as a slow reaction to errors.
In NVIDIA ConnectX-6 Dx and above, the hardware contains a true real-time clock, in PTP format (UTC/TAI).
Implications:
Timestamps do not require a translation; any application can get an accurate timestamp, without software overhead.
Timestamps are more accurate; also, the control loop is tighter and therefore stabilizes faster
The network interface card (NIC) hardware (HW is aware of the network time and is able to perform "time-aware" operations as described in sections Accurate Scheduling, Packet Pacing and Time-based Steering
In the real-time clock, as described in Figure 3, the PTP servo can be either ptp4l or any other commercial PTP stack. The "discipline" is done using standard Linux APIs or POSIX Clock API.
As a result, each packet is timestamped with the UTC/TAI timescale timestamp, which does not require timestamp conversions by the software.
The following diagrams explain the clock discipline procedure. The HW contains a digital circuitry, called UTC time, described in purple in Figure 3 - HW Clock Discipline, which represents the current time in PTP time format. The UTC time is fed by a clock that is controlled by a digital pll.
The digital pll is fed by the local oscillator, which theoretically, can be unstable, as illustrated in Figure 4 - Disciplined Clock Behavior by the red curve. As a result, the packet timestamp although arriving in real-time format, can have an error due to this clock uncertainty.
PTP packets (sync, delay response, etc.,) arrive at the SW PTP daemon, i.e., ptp4l. The PTP daemon runs a servo calculation, that will identify the clock drift and error, and using the Linux API explained above, discipline the UTC time, by changing its frequency. The result, schematically, is illustrated in Figure 4 - Disciplined Clock Behavior by the purple line, that is tracking the ideal behavior of the clock (in which the leader and the follower are neither synchronized not synthesized).
Timestamp conversion in Software
Timestamp conversion by Hardware (ConnectX-6 Dx)
HW Clock Discipline
Disciplined Clock Behavior
To enable the NIC capability to work as a real-time clock (not backward compatible), please refer to section 4.1 Running ptp4l.