[rbak-nsp] SeOS 12.1.1.12p13 issue

[Соловьёв Роман Анатольевич]

_______________________________________________
redback-nsp mailing list
redback-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/redback-nsp

[Olivier Benghozi]

Hi Roman,

Brandon Leeberg in this ML also recently posted about the same issue (with the same prefix by the way), running SEOS-12.1.1.9 and 12.1.1.12p13. Nothing seems bad with this route.

In fact, I found back a pcap capture (from december 2017) of a BGP session from one of my Juniper MX gears toward a BGP/Netflow collector, where I can see this route.
And I can see after all that there's a difference between your version and what was transmitted by this MX
For the AGGREGATOR attribute, the "partial" bit is at 0 in my capture (meaning tat the attribute is "complete", that is everything is OK), whereas in your case it is set at 1 (so the attribute begins with c0 instead of e0).

In Brandon's case the "partial" bit was also at 1.
So I suppose that this is what the SE code doesn't like.

There's no serious reason for this flag to be set to 1 for this prefix (or it means that a BGP router transmitted this announcement without understanding what AGGREGATOR attribute was, which is ridiculous). That's probably a problem on the originator's side.
But there's no reason for SEOS to consider this attribute as bad (and no reason to close the session since RFC7606, but SEOS is now a dead end).

However I guess that on Brandon case, the sessions was staying alive (juste error messages in the logs)...

Seems like a bug to me, I guess that only an Ericsson TAC engineer could help fix this SEOS BGP piece of code.


Olivier

Hi. Some issue is detected with SeOS version SEOS-12.1.1.12p13-Release
The issue is about BGP protocol handling.
The problem is, that SeOS close a BGP session on receiving mailformed UPDATE message from a peer. The peer is Juniper.
bgp_read_v4_message:11175: NOTIFICATION received from 5.143.236.222 (External AS 48711): code 3 (Update Message Error) subcode 4 (attribute flags error), Data: e0 07 08 00 03 02 Apr 30 09:52:06 2018
bgp neighbor 5.143.236.221
BGP neighbor: 5.143.236.221, remote AS: 12389, external link
Version: 4, router identifier: 178.34.128.3
State: Idle for 00:00:25
Last read 00:00:25, last send 00:00:25
Hold time: configured 180, negotiated 0
Keepalive time: configured 30, negotiated 0
Local restart timer 120 sec, stale route retain timer 180 sec
Received restart timer 0 sec, flag 0x0
Number of hops external BGP neighbor may be away: 1
Minimum time between advertisement runs: 30 secs
Source (local) IP address: 0.0.0.0
Received messages: 0 (0 bytes), notifications: 0, in queue: 0
Sent messages: 0 (0 bytes), notifications: 289, out queue: 0
Last active open: 06:10:23, reason: Have not registered with RIB
Reset count: 289, last reset time: 00:00:25, reset reason: Notification sent (update: attribute flags error)
show bgp neighbor 5.143.236.221 malform update
Apr 30 10:42:23 Malformed UPDATE msg (nbr 5.143.236.221, context 0x40080002, 80 bytes, repeated 1512 times, reason: Invalid msg) -
ffff ffff ffff ffff ffff ffff ffff ffff 0050 0200 0000 3540 0101 0040 020e 0203 0000 3065 0000 0c97 0003 02ed 4003 0405 8fec dd40 0600 e007 0800 0302 ed5b dc3f 01c0 0808 3065 0006 3065 0007 185b dc3f
Lets parse this data.
ffff ffff ffff ffff ffff ffff ffff ffff - the init marker
0050 - totak message length - 80 bytes
02 - UPDATE
0000 Length of Withdrawn Routes
0035 Total size of attributes (53 bytes)
40 01 01 00
ORIGIN (IGP)
40 02 0e 02 03 0000 3065 0000 0c97 0003 02ed
40-flags
02 - AS_PATH
0e - length - 14 bytes
02 - segment type AS_SEQUENCE
03 - 3 AS length
0000 3065 0000 0c97 0003 02ed - ASN itself (12389,3223,197357)
40 03 04 05 8f ec dd
NEXT_HOP 5.143.236.221
40 06 00
an empty ATOMIC_AGGREGATE attribute
e0 07 08 0003 02ed 5b dc 3f 01
AGGREGATOR AS 197357 IP 93.220.63.1
c0 08 08 3065 0006 3065 0007
COMMUNITY 12389:6 12389:7
18 5b dc 3f
Prefixes 91.220.63.0/24 <http://91.220.63.0/24>
e0 07 08 0003 02ed 5b dc 3f 01
I don't see anything wrong with it.
IMHO the AGGRETATOR attribute is composed with all RFC requirements
Can somebody explain me such unexpected behavior?

[Marcin Kuczera]

Post by Olivier Benghozi
Hi Roman,
Brandon Leeberg in this ML also recently posted about the same issue
(with the same prefix by the way), running SEOS-12.1.1.9
and 12.1.1.12p13. Nothing seems bad with this route.
In fact, I found back a pcap capture (from december 2017) of a BGP
session from one of my Juniper MX gears toward a BGP/Netflow
collector, where I can see this route.
And I can see after all that there's a difference between your version
and what was transmitted by this MX
For the AGGREGATOR attribute, the "partial" bit is at 0 in my capture
(meaning tat the attribute is "complete", that is everything is OK),
whereas in your case it is set at 1 (so the attribute begins with c0
instead of e0).
In Brandon's case the "partial" bit was also at 1.
So I suppose that this is what the SE code doesn't like.
There's no serious reason for this flag to be set to 1 for this prefix
(or it means that a BGP router transmitted this announcement without
understanding what AGGREGATOR attribute was, which is ridiculous).
That's probably a problem on the originator's side.
But there's no reason for SEOS to consider this attribute as bad (and
no reason to close the session since RFC7606, but SEOS is now a dead end).

This is also what comes from our analyse.
It looks like an implementation bug.

The biggest issue that SEOS is in "end of maintenance state". However,
they still release some newer patches (12.1.1.12p14)..

I have sent this info to a person who worked as routing chief in
Ericsson some time ago, maybe he has some actual contact... but no
response for now...

We are also facing lot of EPPA3 crashes and have no idea what to turn off...

Regards,
Marcin

Post by Olivier Benghozi
However I guess that on Brandon case, the sessions was staying alive
(juste error messages in the logs)...
Seems like a bug to me, I guess that only an Ericsson TAC engineer
could help fix this SEOS BGP piece of code.
Olivier

On 30 apr. 2018 at 23:54, СПлПвьёв РПЌаМ АМатПльевОч
Hi. Some issue is detected with SeOS version
SEOS-12.1.1.12p13-Release
The issue is about BGP protocol handling.
The problem is, that SeOS close a BGP session on receiving
mailformed UPDATE message from a peer. The peer is Juniper.
bgp_read_v4_message:11175: NOTIFICATION received from
5.143.236.222 (External AS 48711): code 3 (Update Message Error)
subcode 4 (attribute flags error), Data:  e0 07 08 00 03 02 Apr
30 09:52:06 2018
*On SeOS side:*
bgp neighbor 5.143.236.221
BGP neighbor: 5.143.236.221, remote AS: 12389, external link
  Version: 4, router identifier: 178.34.128.3
  State: Idle for 00:00:25
  Last read 00:00:25, last send 00:00:25
  Hold time: configured 180, negotiated 0
  Keepalive time: configured 30, negotiated 0
  Local restart timer 120 sec, stale route retain timer 180 sec
  Received restart timer 0 sec, flag 0x0
  Number of hops external BGP neighbor may be away: 1
  Minimum time between advertisement runs: 30 secs
  Source (local) IP address: 0.0.0.0
  Received messages: 0 (0 bytes), notifications: 0, in queue: 0
  Sent messages: 0 (0 bytes), notifications: 289, out queue: 0
  Last active open: 06:10:23, reason: Have not registered with RIB
N*otification sent (update: attribute flags error)*
show bgp neighbor 5.143.236.221 malform update
Apr 30 10:42:23 Malformed UPDATE msg (nbr 5.143.236.221, context
0x40080002, 80 bytes, repeated 1512 times, reason: Invalid msg) -
ffff ffff ffff ffff ffff ffff ffff ffff 0050 0200 0000 3540 0101
0040 020e 0203 0000 3065 0000 0c97 0003 02ed 4003 0405 8fec dd40
0600 e007 0800 0302 ed5b dc3f 01c0 0808 3065 0006 3065 0007 185b dc3f
Lets parse this data.
ffff ffff ffff ffff ffff ffff ffff ffff - the init marker
0050 - totak message length - 80 bytes
*02* -  UPDATE
*0000* Length of Withdrawn Routes
*0035* Total size of attributes (*53 bytes*)
*40 01 01 00*
ORIGIN (IGP)
*40 02 0e 02 03 0000 3065 0000 0c97 0003 02ed*
40-flags
02 -  AS_PATH
0e - length  - 14 *bytes
*
02 - segment type AS_SEQUENCE
03  - 3 AS length
0000 3065 0000 0c97 0003 02ed -  ASN itself (12389,3223,197357)
*40 03 04 05 8f ec dd
*NEXT_HOP**5.143.236.221*
*
*
*
*40 06 00
*an empty ATOMIC_AGGREGATE attribute
*e0 07 08 0003 02ed   5b dc 3f 01 *
AGGREGATOR AS 197357 IP 93.220.63.1
*c0 08 08 3065 0006 3065 0007
*
COMMUNITY 12389:6 12389:7*
*
*18 5b dc 3f
*
Prefixes*  *91.220.63.0/24 <http://91.220.63.0/24>*
*
According the notification message  SeOS threats the AGGREGATOR
*e0 07 08 0003 02ed   5b dc 3f 01 *
I don't see anything wrong with it.
IMHO the AGGRETATOR attribute is composed with all RFC requirements
Can somebody explain me such unexpected behavior?

_______________________________________________
redback-nsp mailing list
https://puck.nether.net/mailman/listinfo/redback-nsp

--
Marcin Kuczera / Wiceprezes Zarządu / CTO
[phone]/ ***@leon.pl <mailto:***@leon.pl>

Leon Sp. z o.o.
ul. Kilińskiego 33d, 44-200 Rybnik
http://www.leon.pl/

INTERNET | TELEWIZJA | TELEFON

KRS 0000223101 Sąd Rejonowy w Gliwicach
Kapitał zakładowy 576.700 zł
NIP: 6332068698

[Olga Savinovskaya]

Does someone have official support with *ML4(service contract)* to ask
Ericsson to realize RFC7606 on SE1200 or SE600?

Our support says that they don't have this contract and that's why they
can not create a ticket about this bug. Our OS Version -
SEOS-12.1.1.11p7-Release and SEOS-12.1.1.12p13-Release

And we have this problem frequently:

Dec 21 08:32:56 notification msg sent (nbr 87.245.245.128, context
0x4008010a 32 bytes, repeated 41 times, code 3/4 (update: attribute
flags error) -  0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020
0303 04e0 0708 0003 0fcb
Jan 30 02:34:31 notification msg sent (nbr 87.245.245.128, context
0x4008010a 32 bytes, repeated 75 times, code 3/4 (update: attribute
flags error) -  0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020
0303 04e0 0708 0003 0fcb
Feb 8  02:34:25 notification msg sent (nbr 87.245.245.128, context
0x4008010a 32 bytes, repeated 82 times, code 3/4 (update: attribute
flags error) -  0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020
0303 04e0 0708 0003 0fcb
Feb 15 04:15:58 notification msg sent (nbr 87.245.245.128, context
0x4008010a 32 bytes, repeated 11 times, code 3/4 (update: attribute
flags error) -  0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020
0303 04e0 0708 0003 0fcb
Feb 24 18:27:19 notification msg sent (nbr 87.245.245.128, context
0x4008010a 32 bytes, repeated 4 times, code 3/4 (update: attribute flags
error) -  0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020 0303
04e0 0708 0003 0fcb
Apr 10 05:57:10 notification msg sent (nbr 87.245.245.128, context
0x4008010a 32 bytes, repeated 180 times, code 3/4 (update: attribute
flags error) -  0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020
0303 04e0 0708 0003 0fcb
Apr 18 00:48:25 notification msg sent (nbr 87.245.245.128, context
0x4008010a 32 bytes, repeated 11 times, code 3/4 (update: attribute
flags error) -  0000 0000 ffff ffff ffff ffff ffff ffff ffff ffff 0020
0303 04e0 0708 0003 0fcb

By the way did someone try to block those AS with incoming route maps ?

_________________
Pozdrawiam, Savinovskaya Olga

--- Treść przekazanej wiadomości ---
Temat: Re: [rbak-nsp] SeOS 12.1.1.12p13 issue
Data: Fri, 11 May 2018 10:37:12 +0200

Post by Olivier Benghozi
Hi Roman,
Brandon Leeberg in this ML also recently posted about the same issue
(with the same prefix by the way), running SEOS-12.1.1.9
and 12.1.1.12p13. Nothing seems bad with this route.
In fact, I found back a pcap capture (from december 2017) of a BGP
session from one of my Juniper MX gears toward a BGP/Netflow
collector, where I can see this route.
And I can see after all that there's a difference between your
version and what was transmitted by this MX
For the AGGREGATOR attribute, the "partial" bit is at 0 in my capture
(meaning tat the attribute is "complete", that is everything is OK),
whereas in your case it is set at 1 (so the attribute begins with c0
instead of e0).
In Brandon's case the "partial" bit was also at 1.
So I suppose that this is what the SE code doesn't like.
There's no serious reason for this flag to be set to 1 for this
prefix (or it means that a BGP router transmitted this announcement
without understanding what AGGREGATOR attribute was, which is
ridiculous). That's probably a problem on the originator's side.
But there's no reason for SEOS to consider this attribute as bad (and
no reason to close the session since RFC7606, but SEOS is now a dead end).

This is also what comes from our analyse.
It looks like an implementation bug.
The biggest issue that SEOS is in "end of maintenance state". However,
they still release some newer patches (12.1.1.12p14)..
I have sent this info to a person who worked as routing chief in
Ericsson some time ago, maybe he has some actual contact... but no
response for now...
We are also facing lot of EPPA3 crashes and have no idea what to turn off...
Regards,
Marcin

Post by Olivier Benghozi
However I guess that on Brandon case, the sessions was staying alive
(juste error messages in the logs)...
Seems like a bug to me, I guess that only an Ericsson TAC engineer
could help fix this SEOS BGP piece of code.
Olivier

On 30 apr. 2018 at 23:54, СПлПвьёв РПЌаМ АМатПльевОч
Hi. Some issue is detected with SeOS version
SEOS-12.1.1.12p13-Release
The issue is about BGP protocol handling.
The problem is, that SeOS close a BGP session on receiving
mailformed UPDATE message from a peer. The peer is Juniper.
bgp_read_v4_message:11175: NOTIFICATION received from
5.143.236.222 (External AS 48711): code 3 (Update Message Error)
subcode 4 (attribute flags error), Data:  e0 07 08 00 03 02 Apr
30 09:52:06 2018
*On SeOS side:*
bgp neighbor 5.143.236.221
BGP neighbor: 5.143.236.221, remote AS: 12389, external link
  Version: 4, router identifier: 178.34.128.3
  State: Idle for 00:00:25
  Last read 00:00:25, last send 00:00:25
  Hold time: configured 180, negotiated 0
  Keepalive time: configured 30, negotiated 0
  Local restart timer 120 sec, stale route retain timer 180 sec
  Received restart timer 0 sec, flag 0x0
  Number of hops external BGP neighbor may be away: 1
  Minimum time between advertisement runs: 30 secs
  Source (local) IP address: 0.0.0.0
  Received messages: 0 (0 bytes), notifications: 0, in queue: 0
  Sent messages: 0 (0 bytes), notifications: 289, out queue: 0
  Last active open: 06:10:23, reason: Have not registered with RIB
N*otification sent (update: attribute flags error)*
show bgp neighbor 5.143.236.221 malform update
Apr 30 10:42:23 Malformed UPDATE msg (nbr 5.143.236.221, context
0x40080002, 80 bytes, repeated 1512 times, reason: Invalid msg) -
ffff ffff ffff ffff ffff ffff ffff ffff 0050 0200 0000 3540 0101
0040 020e 0203 0000 3065 0000 0c97 0003 02ed 4003 0405 8fec dd40
0600 e007 0800 0302 ed5b dc3f 01c0 0808 3065 0006 3065 0007 185b dc3f
Lets parse this data.
ffff ffff ffff ffff ffff ffff ffff ffff - the init marker
0050 - totak message length - 80 bytes
*02* -  UPDATE
*0000* Length of Withdrawn Routes
*0035* Total size of attributes (*53 bytes*)
*40 01 01 00*
ORIGIN (IGP)
*40 02 0e 02 03 0000 3065 0000 0c97 0003 02ed*
40-flags
02 -  AS_PATH
0e - length  - 14 *bytes
*
02 - segment type AS_SEQUENCE
03  - 3 AS length
0000 3065 0000 0c97 0003 02ed -  ASN itself (12389,3223,197357)
*40 03 04 05 8f ec dd
*NEXT_HOP**5.143.236.221*
*
*
*
*40 06 00
*an empty ATOMIC_AGGREGATE attribute
*e0 07 08 0003 02ed   5b dc 3f 01 *
AGGREGATOR AS 197357 IP 93.220.63.1
*c0 08 08 3065 0006 3065 0007
*
COMMUNITY 12389:6 12389:7*
*
*18 5b dc 3f
*
Prefixes**91.220.63.0/24 <http://91.220.63.0/24>*
*
According the notification message SeOS threats the AGGREGATOR
*e0 07 08 0003 02ed   5b dc 3f 01 *
I don't see anything wrong with it.
IMHO the AGGRETATOR attribute is composed with all RFC requirements
Can somebody explain me such unexpected behavior?

_______________________________________________
redback-nsp mailing list
https://puck.nether.net/mailman/listinfo/redback-nsp

--
Marcin Kuczera / Wiceprezes Zarządu / CTO
Leon Sp. z o.o.
ul. Kilińskiego 33d, 44-200 Rybnik
http://www.leon.pl/
INTERNET | TELEWIZJA | TELEFON
KRS 0000223101 Sąd Rejonowy w Gliwicach
Kapitał zakładowy 576.700 zł
NIP: 6332068698