Command disabled: backlink

OSPF Issues

Because Network become rathe large and VPN-channels are rather slow OSPF was adopted as dynamic routing protocol for whole United Networks. OSPF provides Site-to-Site connectivity. It has a number of characteristics:

  • OSPF is implemented by Quagga package. There must be allowed daemons “zebra” and actually “ospfd”:


  • Area is backbone one. It extends from srvgate.gogolya.pushkino to all other nearest (zero-hop) Srvgates (see General Topology).
  • Each Srvgate except Central one has own area behind it (all Subordinate Networks are placed in this area).
  • All Sites' area numbers are combined from the second and the third octet of Srvgate's IP-address in VPN network. For example: srvgate.sokol.msk has address, so its area will be
  • If somewhere there is subordinate connection of one Srvgate to another and then the latter to Central Site from OSPF's viewpooint intermediate area becomes transit area with OSPF-tunneling. It is allowed.
  • Backbone VPN network where Srvgates pick up VPN connections to Central Site there is NBMA (Non-broadcast multi-access) media. So “neighbor” directive must present in ospfd.conf in “router” section.


interface eth0
interface eth1
interface lo
interface tun0
 ip ospf network non-broadcast
router ospf
 network area
 network area
  • Each Srvgate must set up two neighbor relationships: “srvgate.gogolya.pushkino” (DR, router-id and “srvgate.sokol.msk” (BDR, router-id
  • All intra-area routes are aggregated at some Srvgate and announced to core (area as a single “line”.
  • OSPF in United Networks has standard timers and all other parameters.

RIP Issues

A Site where network hardware which can not serve default route presents it is allowed to deploy RIPv2 and implement one-direction redistribution from OSPF to RIP. RIP will be localized at the Site boundary (or in the certain Subordinate Network of this Site) and RIP-compatible hardware will accept routes to all “dust corners” :-) of United Networks - to allow System Administrators to control it. For example DSL-524T has no opportunity to enter manually Default route. So it isn't able to “understand” from where some requests has arrived. This DSL-modem can be seen from local Srvgate only via Web interface (via ARP). There is no way to directly connect it from remote Site. But fortunately it supports RIPv2. So problem can be solved by two methods: via redistribution of all routes from OSPF to RIP and then RIP will announce them to DSL-524T, or simply originate Default route from Srvgate and spread it between RIP-devices. Indeed this is at the discretion of System Administrator responsible for this Site.

BGP Issues

United Networks got Autonomous System in RIPE NCC (AS197627) and Provider Independent IP-block AS/PI and BGP are currently deployed only on Central Site. UN gets two FullViews from two ISPs: Beirel-Telecom and Uniline and announces itself back to them. UN is classified as a stub network.

Since both of operators have uplinks to the sole Higher ISP named “Netorn” there will be the only route on the nearest Internet Exchange “MSK-IX”. By default (in normal situation) this announce is from Beirel due to some routing policies adopted in Netorn. Announce of will be from Uniline in a case of unavailability of Beirel and if the first is still alive. There is no opportunity to load-balance incoming traffic between two United Networks uplinks.

In the future Uniline promises to connect to other ISP in addition to Netorn and then possibility to use different BGP policies for different traffic flows will appear.

But we still can load-balance outgoing traffic. The division of Internet on two halves is used. Let's consider it in detail. Well, let's begin with layout.

As you can see it rather complex for those tasks which it decides. It was done intentionally for educational purpose. First of all AS Number 197627 is 32-bit long. And we wanted to see how Cisco and Quagga (running under FreeBSD) behave. Cisco required Software upgrade immediately. Syntax of BGP clauses differs from Cisco to Quagga. Cisco uses RFC 5396 compliant ASDot Notation where ASN=197627 is represented as ASN32/65536 + ”.” + ASN32%65536, i. e. “3.1019” (representation by two 16-bit numbers). Another subject to study was ASN=23456 that masquerade 32-bit ASNs for systems which “understand” only native 16-bit ASNs.

The second thing was probing iBGP along with eBGP - there is a need in two active routers. Cisco and FreeBSD are two very different types of router. Moreover everyday use of Cisco does not allow to forget Cisco configuration skills. For example ACLs, “sh ip route” and so on.. :-)

The third thing is DMZ organization where UN servers grant protected access to themselves from outside.

The fourth thing is redundancy: to use different hardware to avoid simultaneous failure of two servers and provide to internal users of Site “gogolya.pushkino” stable Internet connection.

So if you look at drawing you can see the following:

  • Srvgate (FreeBSD) has BGP link to Beirel. It is “multihop” eBGP.
  • Sr-2821 (Cisco) has BGP link to Uniline, it is also “multihop” eBGP.
  • Also there is iBGP connection between Srvgate and Sr-2821 through DMZ (once more cause to refresh access lists and firewall rules).
  • Since AS has two uplinks then all internal Vlans have two exits to Internet. There was a reason to set up VRRP between Cisco and FreeBSD for internal Vlans (subnets). VRRP was choosen because it is just one standard based Gateway Redundancy Protocol. HSRP and GLBP supported by Cisco are propreitary protocols. FreeBSD has FreeVRRPD implementation, Cisco has native VRRP. FreeBSD is primary VRRP gateway for internal subnets.
  • Due to eBGP routes received from Beirel are more preferable for FreeBSD then iBGP routes received from Sr-2821 and finally from Uniline. All outgoing traffic will be directed by our AS197627 to Beirel by default. Beirel became primary gateway to Internet for Central Site for different reasons (the first is Netorn and the second is our inner topology).

So, now let's look at prefixies which usual BGP border receives from its neighbors:

/root> netstat -rn
Routing tables

Destination        Gateway            Flags    Refs      Use  Netif Expire         91.223.---.---       UG1         0        0 vlan--        91.223.---.---       UG1         0        0 vlan--        91.223.---.---       UG1         0        0 vlan--        91.223.---.---       UG1         0        0 vlan--       91.223.---.---       UG1         0        0 vlan--        91.223.---.---       UG1         0        0 vlan--         91.223.---.---       UG1         0        0 vlan--         91.223.---.---       UG1         0        0 vlan--
...     91.223.---.---       UG1         0        0 vlan--     91.223.---.---       UG1         0        0 vlan--     91.223.---.---       UG1         0        0 vlan--     91.223.---.---       UG1         0        0 vlan--       89.222.---.---       UG1         0        0 vlan--      89.222.---.---       UG1         0        0 vlan--       89.222.---.---       UG1         0        0 vlan--       89.222.---.---       UG1         0        0 vlan--
...    89.222.---.---       UG1         0        0 vlan--   89.222.---.---       UG1         0        0 vlan--   89.222.---.---       UG1         0        0 vlan--   89.222.---.---       UG1         0        0 vlan--

These are fragments of Full View (or Full Table) with most of routes omitted. The first column has prefixes as increasing sequence:,,…,… Notice that for routes beginning with first octet lower 127 one gateway is used and for routes beginning with first octet greater than 127 is another gateway (91.223.—.— and 89.222.—.—). Mask which has the most significant (and single) bit set to “1” (and therefore it has length of 1 bit) divides this series on two subsequences: and Using the following prefix-lists and route-maps construction:

ip prefix-list Pfx-In seq 5 permit le 32

route-map RM-In permit 10
 match ip address prefix-list Pfx-In
 set local-preference 110
route-map RM-In permit 20
 match as-path 1

…on Uniline uplink we mark the first half of incoming routes on this connection (which will be used later for outgoing traffic from our AS) with higher local-preference then others including all routes incoming from Beirel. These marked routes will be used by the procesing router to choose the first uplink for routes unconditionally and all other routes will be analyzed and they will choose one of two uplinks depending all other BGP attributes. In our certain case it will always be Beirel (see above).

Pay attention to the syntax of previously mentioned construction:

ip prefix-list Pfx-In seq 5 permit le 32

Some comment is needed.

  • ip prefix-list” - it is clause defenition in Cisco/Quagga notation which defines prefix list comparision.
  • seq 5” - sequence number. Prefix list can contain many operation of comparision. First match breaks compare procedure. So position defined by sequence number takes not the least.
  • permit” - positive match (exits if match).
  • le 32” - key phrase! if we would write simply “” it would mean “exact match prefix”. In other words: “match if exactly such prefix”. That is most significant bit is set to “0” and all others have no mean. And mask exactly: 10000000.00000000.00000000.00000000 (in binary). If use “le 32” (“less or equal 32 bit”) then we talk to it: “this prefix and all prefixes with all possible masks 2, 3, 4, … 32 length including”.

Structure of route map is rather transparent: first we check current prefix against defined above prefix list and if match occurs set greater LocalPref to this route (prefix). If no - next operator permits all with defaults (let's remember about default implicit “deny” at the end of route map).

So we can crystalize a “solid”. If current target IP in packet addressed from UN to Internet has the first octet is less than 128 it travels through Uniline (because it marked with LocalPref=110 just on that uplink), for all others will be used Beirel. It is about outgoing traffic. As for incoming one here is Beirel rule. :-)

PBR Issues

Policy Based Routing (PBR) is used at almost all Sites to demarcate Internet route from United Networks. Each Site exits to Internet via local ISP, and when it needs to access some place in UN it uses VPN connection. It's achieved by accepting UN routes via OSPF from Central Site where local Site is connected and pointing Static Default Route (that is less specific) for all other destinations. It is primitive example.

At current moment Central Site uses actively PBR because there are two uplinks. For goals of load-balancing between two uplinks considered Srvgate doesn't use Default route instead of that it divides whole Internet on two halves by using the following routes: and (these both replace Thus when Srvgate initiates connection with some Site somewhere in Internet it looks at address of destination compares it with these two routes and decides which uplink will be used. Answer will arrive on the same interface. So we use both ISPs approximately equally. When some computer in Internet initiates connection to srvgate.gogolya.pushkino DNS-system resolves symbolic name cyclicly into one or anothe address. So here equal use of both uplinks takes place too. This distinctly could be seen when torrents work - there are a lot of peering connection initiated and both uplinks are “chock-a-block” utilized.

When this Srvgate wants to connect to some Site in UN it uses some “more specific route” then above ( and which is received via OSPF from that desired site. This is fully similar other Sites.

Balancing is done on FreeBSD Firewall PF by using Packet Filter's directives “route-to”.

Sometimes particular case of PBR such as Static Routes can be met at work computers of System Administrators. Most of all work computer must participate in Corporate Network (not in United Networks), so default route points to local gateway. But because address pool is used rarely there will not be intersections if we enter static route pointing to via Tunnelling VPN interface. When System Administrator's work computer wants to access Corporate Network or Internet it goes via its Default Route which is provided by local conditions. If work computer needs to access some UN resource it uses VPN connection to Central Site.

OpenVPN Issues

OpenVPN has some issues related to routing process. OpenVPN is a Client-Server pplication. Its daemon on the Server side has own internal routing table which is populated during VPN connection establishing based on Client Configuration. Administrator must manually specify subnets which is connected to VPN-client. Without this information OpenVPN does not provide third-level connectivity even if routing tables on Server and Client are correct.

Using this its routing table OpenVPN offers to its clients some option called “client-to-client direct connection” emulating some variety of Broadcast Media. It is not real broadcasting it is simply emulation. Client sends a packet to OpenVPN Server destined to another client using IP of latter. OpenVPN Server looks at destination IP-address understands that this is address of other Client and resends int to real destination. Traceroute shows absense of intermediate hops during such transmission, but packet really visits two points what is visible from Round-Trip Delay (RTD).

This feature leads to incorrect work of OSPF's “point-to-multipoint” mode. Because of this need to point “neighbors” arises. Often OSPF loose its routes from periferal routers.

routing.txt · Last modified: 2016/01/31 11:14 (external edit)
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki