Create Content

image image image image image

On This Page

This feature is currently not supported in Spectrum-2 based switches.

Synchronizing network applications require their wall clock time to be aligned precisely with a reference time source (to the order of micro seconds or less). To achieve such accuracy, the application needs the support of networking HW (switch and adapter card), to provide the means to stamp time-sensitive packets. It also requires a time synchronization protocol which would make use of the HW time stamping to adjust its wall clock time to an accurate clock in the network.

PTP Principles

The basic principle of PTP is as follows: Slave time = master time + propagation delay + offset.

The purpose of the protocol is to align the slave and the master time so that the gap between them is the propagation delay of the packet. Or in other words, the purpose of the protocol is to use the offset to correct the slave time so the offset between the master sending the packet and the slave receiving the packet is the propagation delay.

Master time is sent periodically by a reliable clock source named Master Clock (MC). In a PTP network, one single reference source is elected called Grand Master Clock (GMC). Propagation delay is calculated between each node and the MC by one of the two methods provided by the standard and further explained below.

To reach sub-microsecond resolutions, all the time stamps which record when a packet is sent and received should be done in the HW. This may impose interaction between SW and HW to query the HW time and send follow-up messages. This issue is further explained below in 2 step section.

Assuming that the propagation delay in the network is symmetric, the propagation time is the average time that took the sync and delay req messages to be switched.

Propagation delay = (T4-T1-(T3-T2))/2=(T4-T1+T2-T3)/2

T1 represents the time that the packet left the master which is actually the master time.

The following figure provides an example of the stages required by a slave clock to align its time to the master clock:

The following table presents the PTP message formats:

Message TypeHex ValueClass

Sync

0

Event

Follow-up

8

General

Delay_Req

1

Event

Delay_Resp

9

General

Pdelay_Req

2

Event

Pdelay_Resp

3

Event

Pdelay_Resp follow-up

A

General

Announce

B

General

Signaling

C

General

Management

D

General

Clock Types and Operation Modes

The types of clocks available are as follows:

  • Grand Master Clock (GMC) – the reference time source derived from an accurate clock such as a GNSS driven clock (i.e. GPS, GLONASS, GALILEO)
  • Boundary Clock (BC) – a network device that acts as slave to its master and as master to its slaves. (Mellanox Onyx implements only this)

  • Ordinary Clock (OC) – a clock that operates either as a Master or a Slave. In the case of a slave, the end point whose clock is been synced (normally a host/server).
  • Master Clock (MC) – a clock which operates as a Master and derives its timing capabilities from the clock chain up to the GMC. It typically serves as a port on a BC connected to a host running as a slave.
  • Transparent Clock (TC) – a PTP aware switch capable of measuring the PTP packet switching delay (transient time) and updating the data in the packet. In peer-to-peer (P2P) delay calculation mechanism, a TC device is also required to calculate its delay from the next hop toward the MC and add the value to the switching delay.

Two modes of delay calculations are defined:

  • End-to-End (E2E) – each slave calculates its delay from the MC by running Delay request/ delay response sequence (Mellanox Onyx implements only this)

  • Peer-to-Peer – propagation delay (Pdelay) is calculated periodically on each link between the slave and the MC independently. The time synchronization packet sent from the MC to all the slaves in the network is updated by each of the downstream nodes with both switching delay (the time that the packet traversed the switch) and upstream hop Pdelay.

PTP Domains

A domain consists of one or more PTP devices communicating with each other. PTP domain defines the scope of PTP message communication, state, operations, data sets, and timescale.

Boundary Clock

In a full E2E PTP deployment, the GMC needs to respond to each slave’s delay request message. A normal profile of PTP may require a few delay calculations per second. An average GMC is capable of addressing few thousands of messages per second. This imposes that direct slave/GMC communication limits the number of overall OCs to ~8K. To scale beyond that, there is a need for a hierarchy between the GMC and the slave. This is achieved by implementing BC, either in the TOR switches or on all the switches in the DC.

The following figure shows the master/slave role that a boundary clock implements between the MC and the Slave (OC).

Each BC acts as a slave towards the GMC and as GMC to its local slaves. Although adding a BC device introduces accuracy degradation as explained above, it becomes mandatory when the number of slaves on a single MC exceeds few thousand devices.

Another use of BC is to bridge between networks. When running PTP over native Ethernet packets, to create larger PTP domains, there is a need to bridge between the broadcast domains. This is done by BC switches.

Default PTP Profile Attributes (SMPTE 2059-2)

NameRangeDefault

Announce interval

-3 (0.125s), 1 (2s)

-2 (0.25s)

Announce timeout interval

2, 10

3

Sync interval (logSyncInt)

-7, -1

-3

Delay request interval

logSyncInt, logSyncInt +5

logSyncInt

PTP domain

0, 127

127

Priority 1

0, 255

128

Priority 2

0, 255

128

Configuring PTP

IEEE 1588 Precision Time Protocol (PTP) may be configured either on router or switch interfaces.

To enable PTP on a router interface you could simply enable it on the selected interface.

The process of configuring PTP on a switch interface is slightly different, however. PTP should be enabled on the interface itself as well as on the respective VLAN interface(s).

All PTP configuration for switch interfaces is taken from those defined on the VLAN interface.

Prior to enabling PTP, NTP must be disabled.

To configure PTP on a router interface:

  1. Enable the PTP CLI commands. Run: 

    switch (config) # protocol ptp
  2. Configure the router interface. Run: 

    switch (config) # interface ethernet 1/1 no switchport force
  3. Add the primary IP address. Run: 

    switch (config) # interface ethernet 1/1 ip address 172.16.1.1/24
  4. Enable PTP on the interface. Run: 

    switch (config) # interface ethernet 1/1 ptp enable

To verify the PTP configuration: 

switch (config) # show ptp
PTP mode                : Boundary Clock
Message format          : Mixed
Acceptable Master Table : Enabled
Domain                  : 127
Clock identity          : 7C:FE:90:FF:FE:FA:21:88
GMC identity            : 7C:FE:90:FF:FE:FA:21:88
Number of master ports  : 1
Slave port interface    : N/A
 
PTP enabled interfaces:
  ----------------------------------------------------
  Port           VLAN      State        Forced Master
  ----------------------------------------------------
  Eth1/1         N/A       MASTER       no

To configure PTP on a switch interface:

  1. Enable the PTP CLI commands. Run: 

    switch (config) # protocol ptp
  2. Add the VLANs. Run: 

    switch (config) # vlan 2-3
  3. Configure VLAN membership.
    For access interfaces, run: 

    switch (config) # interface ethernet 1/2 switchport mode access
    switch (config) # interface ethernet 1/2 switchport access vlan 2

    For trunked interfaces, run: 

    switch (config) # interface ethernet 1/1 switchport mode trunk
  4. Enable PTP on the VLAN interface. Run: 

    switch (config) # interface vlan 2 ptp enable
    switch (config) # interface vlan 3 ptp enable
  5. Enable PTP on the interface. Run: 

    switch (config) # interface ethernet 1/1 ptp enable

    The interface must be a member of the PTP enabled VLAN(s).

To verify the PTP configuration: 

switch (config) # show ptp
PTP mode                : Boundary Clock
Message format          : Mixed
Acceptable Master Table : Enabled
Domain                  : 127
Clock identity          : 7C:FE:90:FF:FE:FA:21:88
GMC identity            : 7C:FE:90:FF:FE:FA:21:88
Number of master ports  : 2
Slave port interface    : N/A
 
PTP enabled interfaces:
  ----------------------------------------------------
  Port           VLAN      State        Forced Master
  ----------------------------------------------------
  Eth1/1         2         MASTER       no
  Eth1/2         2         MASTER       no
  Eth1/1         3         SLAVE        no

Securing PTP Infrastructure

To protect the switch from rogue or mis-configured PTP endpoints, you may secure your Boundary Clock ports by creating an Acceptable Master Table (AMT) and configuring known PTP ports to always behave as a master port via the Forced Master option.

The AMT is a whitelist of up to 8 clock identities that are admissible to take part as valid GrandMasters in the Best Master Clock Algorithm (BMCA).

The Forced Master is enabled on a per-port basis to prevent processing announce messages from a PTP endpoint connected to it, in order for it to always stay in a Master state.

To configure Forced Master on a switch interface, you must enable it on the interface itself as well as on the respective VLAN interface(s).

To configure Acceptable Master Table, add the validated clock identities: 

switch (config) # ptp amt E4:1D:2D:FF:FE:46:13:88
switch (config) # ptp amt E4:1D:2D:FF:FE:44:23:B7

To verify the Acceptable Master Table configuration: 

switch (config) # show ptp amt
 
Clock Identities:
  E4:1D:2D:FF:FE:44:23:B7
  E4:1D:2D:FF:FE:46:13:88

To enable Forced Master on a router interface: 

switch (config) # interface ethernet 1/2 ptp enable forced-master

To verify PTP configuration: 

switch (config) # show ptp
PTP mode                : Boundary Clock
Message format          : Mixed
Acceptable Master Table : Enabled
Domain                  : 127
Clock identity          : 7C:FE:90:FF:FE:FA:21:88
GMC identity            : 7C:FE:90:FF:FE:FA:21:88
Number of master ports  : 1
Slave port interface    : N/A
 
PTP enabled interfaces:
  ----------------------------------------------------
  Port           VLAN      State        Forced Master
  ----------------------------------------------------
  Eth1/2         N/A       MASTER       yes

To configure Forced Master on a switch interface:

  1. Enable Forced Master on the VLAN interface. Run: 

    switch (config) # interface vlan 2 ptp enable forced-master
  2. Enable Forced Master on the interface. Run: 

    switch (config) # interface ethernet 1/1 ptp enable forced-master

    The interface should be a member in the PTP enabled VLAN(s).

To verify PTP configuration: 

switch (config) # show ptp
PTP mode                : Boundary Clock
Message format          : Mixed
Acceptable Master Table : Enabled
Domain                  : 127
Clock identity          : 7C:FE:90:FF:FE:FA:21:88
GMC identity            : 7C:FE:90:FF:FE:FA:21:88
Number of master ports  : 2
Slave port interface    : N/A
 
PTP enabled interfaces:
  ----------------------------------------------------
  Port           VLAN      State        Forced Master
  ----------------------------------------------------
  Eth1/1         2         MASTER       yes
  Eth1/1         3         SLAVE        no

Forced Master is indicated as “yes” only if enabled on the interface and the corresponding VLAN interface.

Additional Reading and Use Cases

For more information about this feature and its potential applications, please refer to the following:

PTP Commands