Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loopback Interface changes for multi ASIC devices #4825

Merged
merged 4 commits into from
Jul 2, 2020
Merged

Loopback Interface changes for multi ASIC devices #4825

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8f22526
Loopback IP changes for multi ASIC devices
arlakshm Jun 22, 2020
ed18962
Change loopback interface from Loopback1 to Loopback4096
arlakshm Jun 22, 2020
094e52b
Addressing review comments
arlakshm Jul 1, 2020
9920fff
change multi_npu to multi_asic
arlakshm Jul 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion dockers/docker-fpm-frr/frr/bgpd/bgpd.main.conf.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,21 @@
! TSA configuration
!
ip prefix-list PL_LoopbackV4 permit {{ get_ipv4_loopback_address(LOOPBACK_INTERFACE, "Loopback0") | ip }}/32
{% if multi_npu() == true %}
ip prefix-list PL_LoopbackV4 permit {{ get_ipv4_loopback_address(LOOPBACK_INTERFACE, "Loopback1") | ip }}/32
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are currently discussions on adding multiple loopback interfaces to Sonic and if there is a regular "Loopback1", say got added from previous config_db etc, it could result in some conflicts. My suggestion is, can we have it renamed to something specific?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@prsunny , changed from Loopback1 to Loopback4096 for ASIC's loopback interface

{% endif %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need it here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pavel-shirshov, I think this might be needed to support TSA for the multi-asic platforms.
Let me know if my understanding is wrong. I will remove this change

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arlakshm I think you're not going to announce your internal loopbacks at all, right? This prefix-list is used to announce loopback only from the device in TSA mode.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in the latest commit

!
{% if get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback0") != 'None' %}
ipv6 prefix-list PL_LoopbackV6 permit {{ get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback0") | replace('/128', '/64') | ip_network }}/64
{% endif %}
{% if multi_npu() == true %}
{% if get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback1") != 'None' %}
ipv6 prefix-list PL_LoopbackV6 permit {{ get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback1") | replace('/128', '/64') | ip_network }}/64
{% endif %}
{% endif %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need it here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pavel-shirshov, I think this might be needed to support TSA for the multi-asic platforms.
Let me know if my understanding is wrong. I will remove this change

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in the latest commit

!
!
{% if DEVICE_METADATA['localhost']['sub_role'] == 'FrontEnd' %}
{% if multi_npu() == true %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is possible to write {% if multi_npu() %}.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed in the latest commit

route-map HIDE_INTERNAL permit 10
set community local-AS
!
Expand All @@ -37,16 +45,30 @@ router bgp {{ DEVICE_METADATA['localhost']['bgp_asn'] }}
{% endif %}
!
{# set router-id #}
{% if multi_npu() == true %}
bgp router-id {{ get_ipv4_loopback_address(LOOPBACK_INTERFACE, "Loopback1") | ip }}
{% else %}
bgp router-id {{ get_ipv4_loopback_address(LOOPBACK_INTERFACE, "Loopback0") | ip }}
{% endif %}
!
{# advertise loopback #}
network {{ get_ipv4_loopback_address(LOOPBACK_INTERFACE, "Loopback0") | ip }}/32
{% if multi_npu() == true %}
network {{ get_ipv4_loopback_address(LOOPBACK_INTERFACE, "Loopback1") | ip }}/32 route-map HIDE_INTERNAL
{% endif %}
!
{% if get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback0") != 'None' %}
address-family ipv6
network {{ get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback0") | ip }}/64
exit-address-family
{% endif %}
{% if multi_npu() == true %}
{% if get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback1") != 'None' %}
address-family ipv6
network {{ get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback1") | ip }}/64 route-map HIDE_INTERNAL
exit-address-family
{% endif %}
{% endif %}
{% endblock bgp_init %}
!
{% block vlan_advertisement %}
Expand Down
32 changes: 25 additions & 7 deletions src/sonic-config-engine/minigraph.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,15 @@ def parse_asic_png(png, asic_name, hostname):
devices[name] = device_data
return (neighbors, devices, port_speeds)

def parse_loopback_intf(child):
lointfs = child.find(str(QName(ns, "LoopbackIPInterfaces")))
lo_intfs = {}
for lointf in lointfs.findall(str(QName(ns1, "LoopbackIPInterface"))):
intfname = lointf.find(str(QName(ns, "AttachTo"))).text
ipprefix = lointf.find(str(QName(ns1, "PrefixStr"))).text
lo_intfs[(intfname, ipprefix)] = {}
return lo_intfs

def parse_dpg(dpg, hname):
aclintfs = None
mgmtintfs = None
Expand All @@ -269,7 +278,6 @@ def parse_dpg(dpg, hname):
"""
if mgmtintfs is None and child.find(str(QName(ns, "ManagementIPInterfaces"))) is not None:
mgmtintfs = child.find(str(QName(ns, "ManagementIPInterfaces")))

hostname = child.find(str(QName(ns, "Hostname")))
if hostname.text.lower() != hname.lower():
continue
Expand All @@ -290,12 +298,7 @@ def parse_dpg(dpg, hname):
ipprefix = ipintf.find(str(QName(ns, "Prefix"))).text
intfs[(intfname, ipprefix)] = {}

lointfs = child.find(str(QName(ns, "LoopbackIPInterfaces")))
lo_intfs = {}
for lointf in lointfs.findall(str(QName(ns1, "LoopbackIPInterface"))):
intfname = lointf.find(str(QName(ns, "AttachTo"))).text
ipprefix = lointf.find(str(QName(ns1, "PrefixStr"))).text
lo_intfs[(intfname, ipprefix)] = {}
lo_intfs = parse_loopback_intf(child)

mvrfConfigs = child.find(str(QName(ns, "MgmtVrfConfigs")))
mvrf = {}
Expand Down Expand Up @@ -445,6 +448,13 @@ def parse_dpg(dpg, hname):
return intfs, lo_intfs, mvrf, mgmt_intf, vlans, vlan_members, pcs, pc_members, acls, vni
return None, None, None, None, None, None, None, None, None, None

def parse_host_loopback(dpg, hname):
for child in dpg:
hostname = child.find(str(QName(ns, "Hostname")))
if hostname.text.lower() != hname.lower():
continue
lo_intfs = parse_loopback_intf(child)
return lo_intfs

def parse_cpg(cpg, hname):
bgp_sessions = {}
Expand Down Expand Up @@ -818,6 +828,7 @@ def parse_xml(filename, platform=None, port_config_file=None, asic_name=None):
cloudtype = None
hostname = None
linkmetas = {}
host_lo_intfs = None

# hostname is the asic_name, get the asic_id from the asic_name
if asic_name is not None:
Expand Down Expand Up @@ -859,6 +870,7 @@ def parse_xml(filename, platform=None, port_config_file=None, asic_name=None):
else:
if child.tag == str(QName(ns, "DpgDec")):
(intfs, lo_intfs, mvrf, mgmt_intf, vlans, vlan_members, pcs, pc_members, acls, vni) = parse_dpg(child, asic_name)
host_lo_intfs = parse_host_loopback(child, hostname)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I get this change. As far as minigraph is concerned, it is yet another loopback interface, correct? Why do we need to have a special parsing for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I get this change. As far as minigraph is concerned, it is yet another loopback interface, correct? Why do we need to have a special parsing for this?

In the multi ASIC minigraph, each ASIC is modeled as different device. The Loopback0 interface is in the DeviceDataPlaneInfo of the device or host, while the Loopback4096 is in the DeviceDataPlaneInfo of the ASIC. Here we are trying to get Loopback0 interface when generating the configuration of the ASIC.

elif child.tag == str(QName(ns, "CpgDec")):
(bgp_sessions, bgp_asn, bgp_peers_with_range, bgp_monitors) = parse_cpg(child, asic_name)
enable_internal_bgp_session(bgp_sessions, filename, asic_name)
Expand Down Expand Up @@ -922,6 +934,12 @@ def parse_xml(filename, platform=None, port_config_file=None, asic_name=None):
for lo_intf in lo_intfs:
results['LOOPBACK_INTERFACE'][lo_intf] = lo_intfs[lo_intf]
results['LOOPBACK_INTERFACE'][lo_intf[0]] = {}

if host_lo_intfs is not None:
for host_lo_intf in host_lo_intfs:
results['LOOPBACK_INTERFACE'][host_lo_intf] = host_lo_intfs[host_lo_intf]
results['LOOPBACK_INTERFACE'][host_lo_intf[0]] = {}

results['MGMT_VRF_CONFIG'] = mvrf

phyport_intfs = {}
Expand Down
3 changes: 3 additions & 0 deletions src/sonic-config-engine/sonic-cfggen
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ from sonic_device_util import get_machine_info
from sonic_device_util import get_platform_info
from sonic_device_util import get_system_mac
from sonic_device_util import get_npu_id_from_name
from sonic_device_util import is_multi_npu
from config_samples import generate_sample_config
from config_samples import get_available_config
from swsssdk import SonicV2Connector, ConfigDBConnector, SonicDBConfig
Expand Down Expand Up @@ -325,6 +326,8 @@ def main():
env.filters['ip_network'] = ip_network
for attr in ['ip', 'network', 'prefixlen', 'netmask', 'broadcast']:
env.filters[attr] = partial(prefix_attr, attr)
# Pass the is_multi_npu function as global
env.globals['multi_npu'] = is_multi_npu
template = env.get_template(template_file)
print(template.render(sort_data(data)))

Expand Down
16 changes: 12 additions & 4 deletions src/sonic-config-engine/sonic_device_util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,20 @@ def get_npu_id_from_name(npu_name):
else:
return None

def get_asic_conf_file_path(platform):
asic_conf_path_candidates = []
asic_conf_path_candidates.append(os.path.join('/usr/share/sonic/platform', ASIC_CONF_FILENAME))
if platform is not None:
asic_conf_path_candidates.append(os.path.join(SONIC_DEVICE_PATH, platform, ASIC_CONF_FILENAME))
for asic_conf_file_path in asic_conf_path_candidates:
if os.path.isfile(asic_conf_file_path):
return asic_conf_file_path
return None

def get_num_npus():
platform = get_platform_info(get_machine_info())
if not platform:
return 1
asic_conf_file_path = os.path.join(SONIC_DEVICE_PATH, platform, ASIC_CONF_FILENAME)
if not os.path.isfile(asic_conf_file_path):
asic_conf_file_path = get_asic_conf_file_path(platform)
if asic_conf_file_path is None:
return 1
with open(asic_conf_file_path) as asic_conf_file:
for line in asic_conf_file:
Expand Down
40 changes: 36 additions & 4 deletions src/sonic-config-engine/tests/multi_npu_data/sample-minigraph.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -387,12 +387,20 @@
<a:LoopbackIPInterface>
<ElementType>LoopbackInterface</ElementType>
<Name>HostIP</Name>
<AttachTo>Loopback0</AttachTo>
<AttachTo>Loopback1</AttachTo>
<a:Prefix xmlns:b="Microsoft.Search.Autopilot.NetMux">
<b:IPPrefix>8.0.0.0/32</b:IPPrefix>
</a:Prefix>
<a:PrefixStr>8.0.0.0/32</a:PrefixStr>
</a:LoopbackIPInterface>
<a:LoopbackIPInterface>
<Name>HostIP1</Name>
<AttachTo>Loopback1</AttachTo>
<a:Prefix xmlns:b="Microsoft.Search.Autopilot.Evolution">
<b:IPPrefix>FD00:1::32/128</b:IPPrefix>
</a:Prefix>
<a:PrefixStr>FD00:1::32/128</a:PrefixStr>
</a:LoopbackIPInterface>
</LoopbackIPInterfaces>
<ManagementIPInterfaces xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution"/>
<ManagementVIPInterfaces xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution"/>
Expand Down Expand Up @@ -457,12 +465,20 @@
<a:LoopbackIPInterface>
<ElementType>LoopbackInterface</ElementType>
<Name>HostIP</Name>
<AttachTo>Loopback0</AttachTo>
<AttachTo>Loopback1</AttachTo>
<a:Prefix xmlns:b="Microsoft.Search.Autopilot.NetMux">
<b:IPPrefix>8.0.0.1/32</b:IPPrefix>
</a:Prefix>
<a:PrefixStr>8.0.0.1/32</a:PrefixStr>
</a:LoopbackIPInterface>
<a:LoopbackIPInterface>
<Name>HostIP1</Name>
<AttachTo>Loopback1</AttachTo>
<a:Prefix xmlns:b="Microsoft.Search.Autopilot.Evolution">
<b:IPPrefix>FD00:2::32/128</b:IPPrefix>
</a:Prefix>
<a:PrefixStr>FD00:2::32/128</a:PrefixStr>
</a:LoopbackIPInterface>
</LoopbackIPInterfaces>
<ManagementIPInterfaces xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution"/>
<ManagementVIPInterfaces xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution"/>
Expand Down Expand Up @@ -526,12 +542,20 @@
<a:LoopbackIPInterface>
<ElementType>LoopbackInterface</ElementType>
<Name>HostIP</Name>
<AttachTo>Loopback0</AttachTo>
<AttachTo>Loopback1</AttachTo>
<a:Prefix xmlns:b="Microsoft.Search.Autopilot.NetMux">
<b:IPPrefix>8.0.0.4/32</b:IPPrefix>
</a:Prefix>
<a:PrefixStr>8.0.0.4/32</a:PrefixStr>
</a:LoopbackIPInterface>
<a:LoopbackIPInterface>
<Name>HostIP1</Name>
<AttachTo>Loopback1</AttachTo>
<a:Prefix xmlns:b="Microsoft.Search.Autopilot.Evolution">
<b:IPPrefix>FD00:3::32/128</b:IPPrefix>
</a:Prefix>
<a:PrefixStr>FD00:3::32/128</a:PrefixStr>
</a:LoopbackIPInterface>
</LoopbackIPInterfaces>
<ManagementIPInterfaces xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution"/>
<ManagementVIPInterfaces xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution"/>
Expand Down Expand Up @@ -580,12 +604,20 @@
<a:LoopbackIPInterface>
<ElementType>LoopbackInterface</ElementType>
<Name>HostIP</Name>
<AttachTo>Loopback0</AttachTo>
<AttachTo>Loopback1</AttachTo>
<a:Prefix xmlns:b="Microsoft.Search.Autopilot.NetMux">
<b:IPPrefix>8.0.0.5/32</b:IPPrefix>
</a:Prefix>
<a:PrefixStr>8.0.0.5/32</a:PrefixStr>
</a:LoopbackIPInterface>
<a:LoopbackIPInterface>
<Name>HostIP1</Name>
<AttachTo>Loopback1</AttachTo>
<a:Prefix xmlns:b="Microsoft.Search.Autopilot.Evolution">
<b:IPPrefix>FD00:4::32/128</b:IPPrefix>
</a:Prefix>
<a:PrefixStr>FD00:4::32/128</a:PrefixStr>
</a:LoopbackIPInterface>
</LoopbackIPInterfaces>
<ManagementIPInterfaces xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution"/>
<ManagementVIPInterfaces xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution"/>
Expand Down
29 changes: 29 additions & 0 deletions src/sonic-config-engine/tests/test_multinpu_cfggen.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,3 +245,32 @@ def test_back_end_asic_acl(self):
argument = "-m {} -p {} -n asic3 --var-json \"ACL_TABLE\"".format(self.sample_graph, self.port_config[3])
output = json.loads(self.run_script(argument))
self.assertDictEqual(output, {})

def test_loopback_intfs(self):
argument = "-m {} --var-json \"LOOPBACK_INTERFACE\"".format(self.sample_graph)
output = json.loads(self.run_script(argument))
self.assertDictEqual(output, {\
"Loopback0": {},
"Loopback0|10.1.0.32/32": {},
"Loopback0|FC00:1::32/128": {}})

# The asic configuration should have 2 loopback interfaces
argument = "-m {} -n asic0 --var-json \"LOOPBACK_INTERFACE\"".format(self.sample_graph)
output = json.loads(self.run_script(argument))
self.assertDictEqual(output, { \
"Loopback0": {},
"Loopback1": {},
"Loopback0|10.1.0.32/32": {},
"Loopback0|FC00:1::32/128": {},
"Loopback1|8.0.0.0/32": {},
"Loopback1|FD00:1::32/128": {}})

argument = "-m {} -n asic3 --var-json \"LOOPBACK_INTERFACE\"".format(self.sample_graph)
output = json.loads(self.run_script(argument))
self.assertDictEqual(output, {\
"Loopback0": {},
"Loopback1": {},
"Loopback0|10.1.0.32/32": {},
"Loopback0|FC00:1::32/128": {},
"Loopback1|8.0.0.5/32": {},
"Loopback1|FD00:4::32/128": {}})