Junos and BGP FlowSpec

Introduction

Recently I carried out tests in labs to evaluate the FlowSpec implementation on MX960 router with TRIO MPC cards. I used a 12.3 Junos release.

Those tests have covered:

-    IPv4 blackholing traffic feature
-    IPv4 rate-limiting traffic feature
-    IPv4 traffic redirection for traffic mitigation (redirect within a VRF)

All tests have been passed with success with just a limitation on traffic redirection presented below. This post presents, how FlowSpec is implemented at RE and PFE level. It presents also the configuration template for Redirecting Traffic through a specific VRF. The “remarking” feature has not been tested.

FlowSpec’s theory

BGP FlowSpec is defined within the RFC 5575: “Dissemination of Flow Specification Rules”. This RFC describes a new NLRI that allows to convey flow specifications (@IP src ; DST ; proto ; port number …) and traffic Action/Rules associated (rate-limiting, redirect, remark …). For IPv4 purposes, RFC defines the following AFI/SAFI value: AFI=1 SAFI=133. FlowSpec uses two existing BGP “containers” to convey both Flow specifications and rules associated.

Indeed, FLOW specifications are encoded within the MP_REACH_NLRI and MP_UNREACH_NLRI attributes. Rules (Actions associated) are encoded in Extended Community attribute.

Within the MP_REACH_NLRI a flow specification is made of a set of components, each component of the flow is combined with each other with an AND operator. Below we describe the different components we have to describe a specific flow (encoded on the NLRI variable field). The Next-Hop length and Next-hop value should be set to 0 (in case of the RFC 5575). Note is should be set other than 0 in this case: BGP Flow-Spec Extended Community for Traffic Redirect to IP Next Hop (draft-simpson-idr-FlowSpec-redirect-02.txt)

NLRI FlowSpec Validation:

 

FlowSpec BGP update received should pass Validation Steps before their installation from the Adj-RIB-IN to LOC-RIB. Next-Hop validation must not be taken into account, because NH FlowSpec (RFC 5575) is always set to 0. But other Validations have to be done before installation (extracted from RFC):

a)    The originator of the flow specification matches the originator of the best-match unicast route for the destination prefix embedded in the flow specification.
b)    There are no more specific unicast routes, when compared with the flow destination prefix, that have been received from a different neighboring AS than the best-match unicast route, which has been determined in step a).

These validations can cause some problems when you use for example external Server to inject FlowSpec updates. Implementations may de-activate these validation steps
 
FlowSpec components ?

Notice from the RFC: “Flow specification components must follow strict type ordering. A given component type may or may not be present in the specification, but if present, it MUST precede any component of higher numeric type value.”

Type 1: Destination prefix component

Type 2: Source prefix component

Type 3: IP Protocol component

  The option byte is defined as following:
-    E bit: end of option (Must be set to 1 for the last Option)
-    A bit: AND bit, if set the operation between several [option/value] is AND, if unset the operation is a logical OR. Never set for the first Option
-    Len: If 0 the following value is encoded in 1 byte ; if 1 the following value is encoded in 2 bytes
-    Lt bit: less than comparison between the Data and the given value
-    Gt bit: greater than comparison between the Data and the given value
-    Eq bit: equal comparison between the Data and the given value

Type 4: Port number component

 Type 5: Destination port number component

 

Type 6: Source port number component

 Type 7: ICMP Type component

Type 8: ICMP Code component

 

Type 9: TCP Flags component

The option byte is defined as following:
-    E bit: end of option (Must be set to 1 for the last Option)
-    A bit: AND bit, if set the operation between several [option/value] is AND, if unset the operation is a logical OR. Never set for the first Option
-    Len: If 0 the following value is encoded in 1 byte ; if 1 the following value is encoded in 2 bytes
-    NOT bit: logical negation operation between Data and the given value
-    m bit: match operation between the Data and the given value

Type 10: Packet Length component

 

Type 11: DSCP Value component

 

Type 12: Fragment component

After the flow definition, Traffic Actions (rules) are encoded as Extended Community Attribute (see RFC 4360)

Extented Community – RFC 4360

There are 4 types of “Action”, each of them has a dedicated Extended Community TYPE. The tab below lists the current Actions available:

Traffic-rate action:

Used for discard or rate-limit a specific flow. Discard action is actually a rate equal to zero. The remaining 4 octets carry the rate (in Bytes/sec) information in IEEE floating point [IEEE.754.1985] format. A nice link to troubleshoot encoded rate can be find here: http://www.h-schmidt.net/FloatConverter/IEEE754.html

Traffic-action action:

Used to trigger specific processing the corresponding flow. Only the last 2 Bits of the 6 bytes are currently defined as following:

 

-    Terminal Action (bit 47): When this bit is set, the traffic filtering engine will apply any subsequent filtering rules (as defined by the ordering procedure). If not set, the evaluation of the traffic filter stops when this rule is applied.
-    Sample (bit 46): Enables traffic sampling and logging for this flow specification.

Redirect action:

Traffic redirection allows to specify a “route-target” community which will be handled by the router to redirect a Flow to a specific VRF.

Traffic-marking action:

Used to force a flow to be re-writted with a specific DSCP value when it leaves the routers.

Flow Specification and JUNOS

Junos allows 2 kind of Flow Routes. First of all, static flow routes which are configured like this:
 

---

#Example: Rate-limiting flow (@IP source 10.1.1.1/32 DNS traffic) at 15Kbits/s
sponge@bob# show routing-options flow
route static-flow1 {
    match {
        source 10.1.1.1/32;
        protocol udp;
        port 53;
    }
    then rate-limit 15k;
}

---

BGP flow routes family is configured as following. Associated to the flow family you can add a policy-statement to disable the validation process of the route (allows direct installation of FlowSpec routes in the BGP LOC-RIB and bypass the FLOW SPEC routes validation process described in chapter 1)
 

---

[edit protocols bgp]
group FLOWSPEC {
    type internal;
    local-address 10.1.1.3;
    family inet {
        unicast;
        flow {
            no-validate NO-VAIDATION;
        }
    }
    neighbor 10.1.1.2 {
        description FS-SERVER;
    }
}

[edit policy-options]
policy-statement NO-VAIDATION {
    term 1 {
        then accept;
    }
}

---

 

Flow routes (BGP or static) are installed within the table “inetflow.0”.

---

sponge@bob> show route table inetflow.0 detail
inetflow.0: 1 destinations, 1 routes (2 active, 0 holddown, 0 hidden)

*,10.1.1.1,proto=17,port=53/term:2 (1 entry, 1 announced)
        *Flow   Preference: 5
                Next hop type: Fictitious
                Address: 0x904d5e4
                Next-hop reference count: 2
                State: <Active>
                Local AS: 65000
                Age: 38
                Validation State: unverified
                Task: RT Flow
                Announcement bits (2): 0-Flow 1-BGP_RT_Background
                AS path: I
                AS path: Recorded
                Communities: traffic-rate:0:1875

---

As I said previously, Next-Hop is always Fictitious (because it is set to 0). The Flow Specification is encoded in an n-turple (the BGP FLOW routes) and rules as an extended-community. Here, the rule type is traffic-rate used for rate-limiting or discarding (rate=0) a specific flow.

Junos then converts this route on a well-known firewall-filter and pushes the filter update on every PFE in both directions (Input / Output). This Firewall Filter is called “__FlowSpec_default_inet__”.

 

This is not an hidden filter and you can have access to it via the cli command:

---

sponge@bob> show firewall filter __FlowSpec_default_inet__

Filter: __FlowSpec_default_inet__
Counters:
Name                                                Bytes              Packets
*,10.1.1.1,proto=17,port=53                             0                    0
Policers:
Name                                                Bytes              Packets
15K_*,10.1.1.1,proto=17,port=53                         0                    0

---

Actually, flow spec firewall filter is a dynamic FW filter which combines in one FW filter all the Flow Specification routes (as a “from” criteria) and their associated rules (as a “then” criteria). Each route can be view as a term.

FlowSpec firewall filter is always at the end of the Firewall Filter list. In other words, you can use before your own Firewall Filter(s) applied in both directions via the command “set interface xxx family inet filter [input|input-list|output|output-list]”. Your FW filter will be always analyzed first before the flow spec FW filter. 

But the flow spec Firewall Filter will be always analyzed even if a previous static firewall filter has a “then” action to ACCEPT. So, you can never bypass the “__FlowSpec_default_inet__” filter. Also remember that FLOW SPEC Firewall Filter is applied in both direction at PFE level.

At PFE Level you can use this command to check how the “__FlowSpec_default_inet__” is programed.

---

ADPC0(bob vty)# show filter
Program Filters:
---------------
   Index     Dir     Cnt    Text     Bss  Name
--------  ------  ------  ------  ------  --------
[…]
   17000      52       0       4       4  __default_arp_policer__
   57008     104     288      16      16  __cfm_filter_shared_lc__
   65024     104     144      36      36  __FlowSpec_default_inet__
   65280      52       0       4       4  __auto_policer_template__
   65281     104       0      16      16  __auto_policer_template_1__
[…]
16777216     104     288      36      36  fnp-filter-level-all

ADPC0(bob vty)# show filter index 65024 program
Program Filters:
---------------
   Index     Dir     Cnt    Text     Bss  Name
--------  ------  ------  ------  ------  --------
   65024     104     144      36      36  __FlowSpec_default_inet__

Action directory: 2 entries (104 bytes)
    0: accept counter 0 policer 0
       -> 7:
    1: accept
       -> 8:(bss location 3:)
Counter directory: 1 entry (144 bytes)
    0: Counter name "*,10.1.1.1,proto=17,port=53": 1 reference
Policer directory: 1 entry (176 bytes)
    0: Policer name "15K_*,10.1.1.1,proto=17,port=53": 1 reference
       Bandwidth Limit: 1875 bytes/sec.
       Burst Size: 15000 bytes.
       discard
Program instructions: 9 words

    0: match protocol != 17 -> 8:
       set icmp-type
       match source-port == 53 -> 5:
       set destination-port
       match destination-port != 53 -> 8:

    5: match source-address != 0x0a010101 -> 8:
       terminate -> action index 0

    8: terminate -> action index 1

---

As presented in chapter 1 ; the second useful Action type is “redirect”. A redirect action is also encoded in a specific extended community. In Junos you can then use this specific community to redirect a specific traffic within a VRF. Very useful for traffic mitigation or traffic inspection (IDP).

A flow route with the redirect action example:
 

---

sponge@bob> show route 10.1.1.0/30 detail

inetflow.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
10.1.1.0,*,proto=17/term:1 (1 entry, 1 announced)
        *BGP    Preference: 170/-101
                Next hop type: Fictitious
                Address: 0x9041484
                Next-hop reference count: 1
                State: <Active Int Ext>
                Local AS: 65000 Peer AS: 65000
                Age: 48
                Validation State: unverified
                Task: BGP_65000.81.253.192.94+31919
                Announcement bits (1): 0-Flow
                AS path: ?
                AS path: Recorded
                Communities: redirect:65000:123456
                Accepted
                Localpref: 100
                Router ID: 81.253.192.94

---

The only limitation on Junos when you redirect traffic is that the server which will be in charge of mitigation or inspection or anything else must not be directly connected to the same router attached to the VRF. Indeed, Junos pushes FlowSpec routes towards all PFE. So if you dynamically redirect a traffic to a server to perform “traffic processing”, the “coming back” traffic will be again redirect and so on… In this configuration you will experience a forwarding loop.

The diagram below shows this limitation and 2 workarounds (design based).

  

On Junos traffic redirection is configured as following:

---

[edit protocols bgp]
group FLOWSPEC {
    type internal;
    local-address 10.1.1.3;
    family inet {
        unicast;
        flow {
            no-validate NO-VAIDATION;
        }
    }
    neighbor 10.1.1.2 {
        description FS-SERVER;
    }
}

[edit policy-options]
policy-statement NO-VAIDATION {
    term 1 {
        from community redirect;
        to instance PROCESSING-VRF;
        then accept;
    }
    term 2 {
        then accept;
    }
}
community redirect membersredirect:65000:123456;

[edit routing-instances]
PROCESSING-VRF {
    instance-type vrf;
    interface ge-2/2/1.0;
    route-distinguisher 12.2.2.2:1234;
    vrf-target target:65000:123456;
    routing-options {
        static {
            defaults {
                resolve;
            }
            # DEFAULT route if Server in charge of processing traffic is directly attached
            # to the VRF
            route 0.0.0.0/0 next-hop 10.128.2.10;
        }
    }
}