Nornir: Inventory Structure and Filtering

The inventory is at the core of Nornir holding all the hosts that tasks will be run against and the variables that will be used by those tasks. Before any tasks can be run by Nornir the inventory has to be initialised.

Table Of Contents

Inventory Structure

The inventory is structured into 3 parts, nr.inventory.hosts, nr.inventory.groups and nr.inventory.defaults. hosts and groups are nested dictionaries defining membership and data with the elements called by host/group name or iteration. This does not apply for defaults as it doesn’t have any actual members so is not nested.

These examples in this post are based on the default SimpleInventory plugin, it is the same logic for all inventory plugins.

nr.inventory.hosts
{'test_host': Host: test_host,
 'test_grp': Host: test_grp,
 'test_default': Host: test_default}

nr.inventory.groups
{'test': Group: test}

nr.inventory.defaults
<nornir.core.inventory.Defaults at 0x109979ee0>

dict() is a special inventory function that displays the original attributes of a host, group or the defaults. It does not show inherited values.

nr.inventory.hosts['test_grp'].dict()
{'name': 'test_grp',
 'connection_options': {},
 'groups': ['test'],
 'data': {},
 'hostname': '172.17.4.2',
 'port': None,
 'username': 'username',
 'password': None,
 'platform': None}

nr.inventory.groups['test'].dict()
nr.inventory.defaults.dict()

The inventory objects are made up of core elements (username, password, etc), data and connection_options dictionaries. Any undefined keys are automatically added with a value of None or an empty dictionary.

Core elements

With the exception of group all of the core elements will inherit their value from groups or defaults. If they are not set in any of these locations they will have a value of None. When calling any of these values it shows the inherited value, so the value that is used by the inventory.

nr.inventory.hosts['test_grp'].name
nr.inventory.hosts['test_grp'].username
nr.inventory.hosts['test_grp'].password
nr.inventory.hosts['test_grp'].port
nr.inventory.hosts['test_grp'].platform
nr.inventory.hosts['test_grp'].hostname
nr.inventory.hosts['test_grp'].group

Connection options

The connection_options dictionary is always present and inherits its contents (username, password, hostname, etc) from the hosts, groups and defaults core elements. For this reasoning there is no need to define these unless using multiple connection plugins (like scrapli and napalm) or there is a requirement to have additional parameters in the extras dictionary (settings such ssh_keys).

Calling connection_options as you would a normal dictionary will not show the inheritance:

nr.inventory.hosts['test_host'].connection_options['scrapli'].dict()
{'extras': None,
 'hostname': None,
 'port': None,
 'username': 'host_conn_opt_user',
 'password': None,
 'platform': None}

You need to use the special get_connection_parameters() function to see the connection_options inheritance:

nr.inventory.hosts['test_host'].get_connection_parameters('scrapli').dict()
{'extras': {},
 'hostname': '172.17.4.1',
 'port': 'default_conn_opt_port',
 'username': 'host_conn_opt_user',
 'password': 'host_core_pass',
 'platform': 'host_core_platform'}

Individual elements from within it can be called in the same manner as core elements:

nr.inventory.hosts['test_host'].get_connection_parameters('scrapli').username
'host_conn_opt_user'

As per the core elements if the connection_option dictionary is undefined it will inherit its values from groups or defaults.

nr.inventory.hosts['test_grp'].get_connection_parameters('scrapli').dict()
{'extras': {},
 'hostname': '172.17.4.2',
 'port': 'default_conn_opt_port',
 'username': 'grp_conn_opt_user',
 'password': 'default_core_pass',
 'platform': 'grp_core_platform'}

Data Dictionary

The data dictionary is the place to store any custom host or group data (variables). It contains non-inventory data that can be used for things such as inventory filtering and device configurations (variables to render templates to produce the config).

nr.inventory.hosts['test_host'].dict()
{'name': 'test_host',
'connection_options': {},
 'groups': ['test'],
 'data': {'test': 'host_data'},
nr.inventory.hosts['test_host'].data
{'test': 'host_data'}

When the inventory is initialised the data dictionary contents are converted into direct dictionaries of the hosts. Only the data dictionary contents are callable in the normal python manner using the dictionary key name, none of the other core inventory elements are callable in this way.

nr.inventory.hosts['test_host'].items()
dict_items([('test', 'host_data')])
nr.inventory.hosts['test_host']['test']
'host_data'

Data dictionary inheritance can be seen by using traditional Python methods such as x[‘name’], .items(), keys() and (values() to call them.

nr.inventory.hosts['test_grp'].data
{}
nr.inventory.hosts['test_grp'].items()
dict_items([('test1', 'grp_data'), ('test', 'default_data')])

Updating the Inventory

Once the inventory is initialised it can be edited or added to in the normal Python manner with inheritance automatic.

Core elements

Any changes made to core elements in hosts, groups or defaults will automatically be inherited by hosts, group members and connection_options on-the-fly. To change any of these values call them using the .core_element format.

nr.inventory.hosts['test_grp'].groups
 [Group: test]
nr.inventory.hosts['test_grp'].username
'grp_core_user'
nr.inventory.groups['test'].username = 'custom_grp_user'
nr.inventory.hosts['test_grp'].username
'custom_grp_user'

This shows how the hosts username is changed and then inherited by connection_options.

nr.inventory.hosts['test_default'].username = 'custom_core_user'
nr.inventory.hosts['test_default'].username
'custom_core_user'
nr.inventory.hosts['test_default'].get_connection_parameters('scrapli').username
'custom_core_user'

Connection options

Connection options can only be updated if that specific connection_options object exists at that level. For example, the scrapli username can only be changed under the host if it exists under the host. If it exists under the group and was inherited then it must be updated in the group and inherited by the host.

nr.inventory.groups['test'].get_connection_parameters('scrapli').password
'default_core_pass'
nr.inventory.hosts['test_grp'].get_connection_parameters('scrapli').password
'default_core_pass'

nr.inventory.groups['test'].connection_options['scrapli'].password = 'grp_conn_opt_password'
nr.inventory.hosts['test_grp'].get_connection_parameters('scrapli').password
'grp_conn_opt_password'

By default the connection_options extras dictionary value is None. The first object added to it must be a dictionary, after that subsequent objects can be added in the usual manner. The extras dictionary as a whole will only be inherited if it doesn’t already exist (value of None), its contents will not be merged. For example, if the extras dictionary is added to a group it will only be inherited by the host if the host does not have its own extras dictionary.

nr.inventory.groups['test'].connection_options['scrapli'].dict()
{'extras': None,
nr.inventory.groups['test'].connection_options['scrapli'].extras = dict(obj1='first_dict')
nr.inventory.groups['test'].connection_options['scrapli'].extras['obj2'] = 'next_dict'

Data Dictionary

As the data dictionary elements become direct host or group variables at inventory initialization any changes made to them are done directly against the host, group or defaults, not the actual data dictionary.

nr.inventory.hosts['test_default']['site'] = 'host_site'
nr.inventory.hosts['test_default']['site']
'host_site'
nr.inventory.hosts['test_default'].data
{'site': 'host_site'}
nr.inventory.hosts['test_default'].dict()
{'name': 'test_default',
 'connection_options': {},
 'groups': [],
 'data': {'site': 'host_site'},

Adding a new dictionary to a group ensures that it is inherited by all members (hosts) of that group.

nr.inventory.groups['test']['site'] = 'grp_site'
nr.inventory.hosts['test_grp']['site']
'grp_site'
nr.inventory.hosts['test_grp'].items()
dict_items([('test1', 'grp_data'), ('site', 'grp_site'), ('test', 'default_data')])

To add or edit a dictionary in defaults it has to be done against .data as there are no actual host or group objects in it.

nr.inventory.defaults.data['site'] = 'default_site'

Create/ update groups

You cannot create groups or amend group membership after inventory initialisation, this need to be done in the inventory files or within an inventory-plugin. This problem was raised as a bug which turned into a feature request but it has not been worked on for a while.

My original thoughts on this were wrong, it is possible to add group membership after inventory initialisation as well as manually update the host_vars with inherited group_vars. However, the inheritance is on a per-host basis as you are manually updating them, I don’t know there is a builtin way to do this for all hosts.

from nornir.core.inventory import Group
nr.inventory.groups["new_grp"] = Group(name="new_grp", data=dict(vlan=10))
nr.inventory.hosts["HME-SWI-VSS01"].groups.add(Group(name="new_grp"))

Although the host is now a member of the group it has not yet inherited the group_vars.

nr.inventory.hosts["HME-SWI-VSS01"].data
{'site': 'london', 'role': 'campus', 'type': 'network_device'}
nr.inventory.hosts["HME-SWI-VSS01"].groups
'groups': ['ios', 'new_grp']
nr.inventory.groups["new_grp"].data
{'vlan': 10}

To update the host_vars you have to update the data dictionary in the same manner as you would when adding elements to a normal dictionary. As the core host elements are mostly strings (username, platform, etc) to update these you have to replace the dictionary value.

nr.inventory.hosts["HME-SWI-VSS01"].data.update(nr.inventory.groups["new_grp"].data)
nr.inventory.hosts["HME-SWI-VSS01"].platform = nr.inventory.groups["new_grp"].platform

nr.inventory.hosts["HME-SWI-VSS01"].data
{'site': 'london', 'role': 'campus', 'type': 'network_device', 'vlan': 10}

Updating username and password

It is highly likely that either of these values will need to be updated after inventory initialisation as users could change and it isn`t a good idea to have clear-text passwords in variable files.

The easiest method to accomplish this is to update defaults as these will be inherited by hosts, groups and all connection_options.

nr.inventory.defaults.username = 'default_username'
nr.inventory.defaults.password = 'default_password'

These could be assigned in a more conditional manner such as to groups or by looping through a subset of hosts.

nr.inventory.groups['iosxe'].username = 'group_username'
nr.inventory.groups['iosxe'].password = 'group_password'

for host_obj in nr.inventory.hosts.values():
    host_obj.username = 'host_username'
    host_obj.password = 'host_password'

Inventory Filtering

Different filtering methods allow for differing levels of complexity in filtering of the inventory of hosts which Nornir tasks will be run against. The inventory can be filtered based on any of the host or group properties. The outcome of the filter is saved to a variable which is then used with the run method to run Nornir tasks against the filtered inventory.

dc_nr = nr.filter(Infra_Location="DC")
dc_nr.run(task=napalm_get, getters=["facts"])

children_of_group is a pre-defined function that allows for easy filtering of group members. It returns a set of hosts (not an inventory) that belongs to a group including those that belong indirectly via inheritance. It cannot be used as an inventory to run Nornir tasks against.
To view or edit the attributes of individual members either iterate through the set or convert it into a list and call the list element. As the output can’t be used as an inventory I haven’t really found a use case for children_of_group yet.

nr.inventory.children_of_group("ios")
{Host: DC-9300-SWI01, Host: HME-SWI-ACC01, Host: HME-SWI-VSS01}

list(nr.inventory.children_of_group("ios"))[0].items()
dict_items([('MachineType', 'Cisco Catalyst 9300 Series Switch'), ('IOSVersion', '16.9.2, RELEASE SOFTWARE (fc4)'), ('Infra_Location', 'DC'), ('type', 'switch')])

There are 3 methods of filtering with Filter Methods being the most basic whilst F object and Filter Functions offer more granularity and complex filtering.

Filter Method (basic)

In its most basic form you can filter on an exact match of any inventory attribute, be it from a host or group. The filtered inventory can be viewed and called as you would the normal inventory.

az = nr.filter(Infra_Location="AZ")
az.inventory.hosts
{'AZ-ASR-WAN01': Host: AZ-ASR-WAN01, 'AZ-FPR-FTD01': Host: AZ-FPR-FTD01, 'AZ-ASA-VPN01': Host: AZ-ASA-VPN01, 'AZ-UBT-SVR01': Host: AZ-UBT-SVR01}

Filters can be chained together to narrow down the results further. It is sequential rather than OR logic so the ordering of the filters is important.

az_routers = nr.filter(Infra_Location="AZ").filter(type="router")
az_routers.inventory.hosts
{'AZ-ASR-WAN01': Host: AZ-ASR-WAN01}

F object (advanced)

F object is a more powerful method of filtering which allows for greater complexity and granularity in the matchings.

Operations are the methods used for matching in each F object statement.

Operators can be used to chain and/or negate F object statements.

F is a separate module that first needs to be imported.

from nornir.core.filter import F

Operations are defined in the format F( dict_key __ operator = condition ) with the operators either before (NOT) or between (AND, OR) the operation. The pattern matching is case sensitive and regex cannot be used inside the operation condition (what is being matched).

nr.filter(~F(platform__eq="junos"))						                        All devices not of a platform "junos"
nr.filter(F(Infra_Location__eq="AZ") & F(type__eq="switch"))					All switches in AZ
nr.filter(F(type__eq="router") & (F(Infra_Location__eq="AZ") | (F(Infra_Location__eq="DC1")))		All routers in AZ or DC1

nr.filter(F(sla__ge=80))					                                    Devices with an sla equal to or greater than 80
nr.filter(F(name__contains="NET"))				                                All devices that have NET in their host name
nr.filter(F(Infra_Location__startswith="DC"))			                        All devices at DC1 or DC2 (start with DC)

nr.filter(F(type__any=["switch", "dc_switch", "firewall"])) 		            Devices that are either a switch, dc_switch or firewall
nr.filter(F(type__in=["switch", "dc_switch", "firewall"]))		                Same result as __any, not sure if is any real difference

nr.filter(F(has_parent_group="ios"))				                            Devices that are in the ios group
nr.filter(F(groups__all=["ios","router"]))			                            Devices must be in all these groups (ios and router)
nr.filter(F(groups__any=["ios","iosxe", "nxos"]))			                    Devices can be in any one of these groups

There is no equivalent of Ansible run_one in Nornir, however you can use filtering to grab the first element from inventory to achieve the same thing. The second example filters the first host from each of these groups, so you could for example render a config snippet for each OS type.

nr.filter(name=list(nr.inventory.hosts.keys())[0])
nr.filter(F(groups__any=['nxos', 'ios', 'iosxe']) & F(name=list(nr.inventory.hosts.keys())[0]))

Filter Functions (advanced)

Filter functions allow the filtering of the nornir inventory through another function providing the flexibility to use any of Pythons existing packages to do the filtering. The filtering logic is written and managed in another function and called when needed.

The filter function loads the host as an argument and checks against each host either returning True or the matching hosts. It is called using filter_func and does not require any arguments to be manually loaded into the function.

For example this uses regex to match hosts which have even numbered hostnames.

import re
def even_device_naming_convention(host):
    if re.match(".*[0-9][2,4,6,8,0]$", host.name):
        return True

nr.filter(filter_func=even_device_naming_convention).inventory.hosts
{'HME-WLC-AIR02': Host: HME-WLC-AIR02, 'DC-UBT-SVR02': Host: DC-UBT-SVR02

This does the same thing but returns host.name instead of True.

def even_device_naming_convention(host):
    if re.match(".*[0-9][2,4,6,8,0]$", host.name):
        return host.name

nr.filter(filter_func=even_device_naming_convention).inventory.hosts
{'HME-WLC-AIR02': Host: HME-WLC-AIR02, 'DC-UBT-SVR02': Host: DC-UBT-SVR02

This is a really good post on Nornir filtering where I got a lot of the information from.

• ← Previous Post
• Next Post →