Generating Configuration and Commands using Jinja2 and Python

In the modern world of network engineering, network automation has become the norm. In my experience, some repetitive configuration can be automated by making a python script that executes the same command through an iteration of device list. Further, the engineer can also opt to use Ansible for this kind of use case.

But not all tasks and workflows would be automated. Some engineering teams still resort to manual configuration by writing the configuration manually then executing it through copy and paste during the change. For this use case, Jinja2 and python are the use cases that I think that could be used.

To make it simple as possible: Jinja2 is a templating language. We could generate text as well in python alone or generate a start up config by working around with an excel file with variables for input. But Jinja2 template can operate with the functions and logic, which makes it more flexible to work with.

3 components: the python script, variable file, and Jinja2 template

Python File

"""
Purpose: Automate the generation of configuration and commands for advertising/withdrawing/transferring prefix 
John Dominic Abat, November 2023
"""

#import modules
#import variable_list, a .py file containing my defined variable values 

from jinja2 import Environment, FileSystemLoader
from variable_list import *


#define the environment, where the template value is rendered 
#template is rendered to the environment and assigned to tpl 
#output contains the text that is generated by the render, where the text is produced from the template and its pre-defined variables. (previously defined in variable_list)

env = Environment(loader=FileSystemLoader("templates"))
tpl = env.get_template("configuration_template.txt")
output = tpl.render(Alloc_IP_block = Alloc_IP_block, Recov_IP_block = Recov_IP_block, mask = mask, src_node = src_node, dst_node = dst_node, method = method, check_commands = check_commands)

print(output)

• this is the python script to execute, comments indicated for each code section

Variable File

Recov_IP_block = ['10.10.10.0', '11.11.11.0']
src_node = 'SOURCE_NODE'

Alloc_IP_block = ['10.10.10.0', '11.11.11.0']
dst_node = 'DESTINATION_NODE'

mask = 21
method = 'transfer'
check_commands = ['sh users | i ','sh run | i ','sh ip local pool | i ']




#Recov_IP_block: list of Prefixes to be recovered
#src_node: node where the IP is going to be removed 

#Alloc_IP_Block: list of Prefixes to be allocated 
#dst_node: node where the IP is going to be configured 

#mask: subnet mask
#method: could be (allocate, recover, transter, swap) - allocate when from IPAM, recover when node to IPAM, transfer when node to another node, swap when an IP from IPAM will replace an IP in node

• Imported by python file. This file contains the defined variables that will be used by the template whenever the variable values are called.
• this will simplify the process since variables are the ones going to be edited per change. Engineer will not repetitively copy and paste the commands then editing out the parts like IP addresses. (for example, BGP network command will be automatically generated. Prefix and mask are arbitrary values. Everything else like route-maps are the same, given that the configuration are standardized across the network.

Template

{% set mask_var = mask -%}
{% set Alloc_IP_Prefix_var = Alloc_IP_block -%}
{% set Recov_IP_Prefix_var = Recov_IP_block -%}
{% set source_device_var = src_node -%}
{% set dest_device_var = dst_node -%}
{% set check_commands_var = check_commands -%}

{% set i = 0 -%}

{% if mask_var == 24 -%}
{% set dotted_var = '255.255.255.0' -%}
{% set stop = 1 -%}
{% elif mask_var == 23 -%}
{% set dotted_var = '255.255.254.0' -%}
{% set stop = 2 %}
{% elif mask_var == 22 %}
{% set dotted_var = '255.255.248.0' -%}
{% set stop = 4 -%}
{% elif mask_var == 21 %}
{% set dotted_var = '255.255.240.0' -%}
{% set stop = 8 -%}
{% endif -%}


================================Implementation================================================
{# ----------start of recovery---------- #}
{% if method == 'transfer' or method == 'recover' or method == 'swap' -%}
-----------------------------Removing IP Local pool-----------------------------

***********************
{{ source_device_var }}
**********************

conf t
{% for IP_Prefix in Recov_IP_Prefix_var -%}
{% set command ='no ip local pool DHCP_NAME ' -%}
{% with y = command -%}
{% for i in range(0, 255) if i<stop -%}
{% set IP_split = IP_Prefix.split('.')[0:3] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set combine_1 = [a,b,c+i,1] -%}
{% set combine_254 = [a,b,c+i,254] -%}
{{ y }}{{ combine_1 | join('.') }} {{ combine_254 | join('.') }}
{% set i = i+1 -%}
{% endfor -%}
{% endwith -%}
{% endfor -%}
exit






----------------------Removing network statement, static route, and Prefix_list-----------------------------

***********************
{{ source_device_var }}
**********************

conf t
router bgp 1234
 address-family ipv4 
{%- for IP_Prefix in Recov_IP_Prefix_var %}
  no network {{ IP_Prefix }} mask {{ dotted_var }} route-map ROUTE_MAP_NAME
{%- endfor %}  
  exit
 exit
{% for IP_Prefix in Recov_IP_Prefix_var-%}
no ip route {{ IP_Prefix }} {{ dotted_var }} Null0
{% set IP_split = IP_Prefix.split('.')[0:4] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set d = IP_split[3] | int -%}
{% set combine_seq = [a,b,c] -%}
no ip prefix-list PREFIX_LIST_NAME seq {{ combine_seq | join() }} permit {{ IP_Prefix }}/{{ mask_var }} le 32
{% endfor -%}
exit
wr


{% endif -%}

{# ----------end of recovery---------- #}

{# ----------start of allocation---------- #}

{%- if method == 'transfer' or method == 'allocate' or method == 'swap' -%}

--------------Configuring Static Route, Prefix-list, network statement, and DHCP-----------------------

***********************
{{ dest_device_var }}
**********************


conf t
{% for IP_Prefix in Alloc_IP_Prefix_var -%}
ip route {{ IP_Prefix }} {{ dotted_var }} Null0
{% set IP_split = IP_Prefix.split('.')[0:4] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set d = IP_split[3] | int -%}
{% set combine_seq = [a,b,c] -%}
ip prefix-list PREFIX_LIST_NAME seq {{ combine_seq | join() }} permit {{ IP_Prefix }}/{{ mask_var }} le 32
{% endfor -%}
router bgp 1234
 address-family ipv4
{%- for IP_Prefix in Alloc_IP_Prefix_var %}
  network {{ IP_Prefix }} mask {{ dotted_var }} route-map ROUTE_MAP_NAME
{%- endfor %}
  exit
 exit
{% for IP_Prefix in Alloc_IP_Prefix_var -%}
{% set command ='ip local pool DHCP_NAME ' -%}
{% with y =command -%}
{% for i in range(0, 255) if i<stop -%}
{% set IP_split = IP_Prefix.split('.')[0:3] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set combine_1 = [a,b,c+i,1] -%}
{% set combine_254 = [a,b,c+i,254] -%}
{{ y }}{{ combine_1 | join('.') }} {{ combine_254 | join('.') }}
{% set i = i+1 -%}
{% endfor -%}
{% endwith -%}
{% endfor -%}
exit
wr








{% endif -%}


{# ----------end of allocation---------- #}






################################################POSTCHECKS################################################

{% if method == 'recover' -%}
*********************
{{ source_device_var }}
*********************

{% for IP_Prefix in Recov_IP_Prefix_var -%}
{% for command in check_commands_var -%}
{% for i in range(0, 255) if i<stop -%}
{% set IP_split = IP_Prefix.split('.')[0:3] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set combine = [a,b,c+i] -%}
{{ command }}{{ combine | join('.') }}
{% endfor -%}
{% endfor -%}
{% endfor -%}

{% for IP_Prefix in Recov_IP_Prefix_var %}
sh ip route {{ IP_Prefix }} {{ dotted_var }}
sh ip route {{ IP_Prefix }} {{ dotted_var }} longer 
{%- endfor %}


*****************
Reference node 
*****************
{% for IP_Prefix in Recov_IP_Prefix_var %}
sh route {{ IP_Prefix }}
sh route longer {{ IP_Prefix }}/{{ mask_var }}
sh bgp {{ IP_Prefix }}
{% endfor %}


*****************
{%- for IP_Prefix in Recov_IP_Prefix_var%}
{%- set IP_split = IP_Prefix.split('.')[0:3] %}
{%- set a = IP_split[0] | int %}
{%- set b = IP_split[1] | int %}
{%- set c = IP_split[2] | int %}
{%- set combine_1 = [a,b,c,1] %}
tracert {{ combine_1 | join('.') }}
{%- endfor %}
*****************



{% endif %}


{% if method == 'allocate' or method == 'transfer' or method == 'swap' -%}
*********************
{{ dest_device_var }}
*********************

{% for IP_Prefix in Alloc_IP_Prefix_var -%}
{% for command in check_commands_var -%}
{% for i in range(0, 255) if i<stop -%}
{% set IP_split = IP_Prefix.split('.')[0:3] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set combine = [a,b,c+i] -%}
{{ command }}{{ combine | join('.') }}
{% endfor -%}
{% endfor -%}
{% endfor %}

{% for IP_Prefix in Recov_IP_Prefix_var %}
sh ip route {{ IP_Prefix }} {{ dotted_var }}
sh ip route {{ IP_Prefix }} {{ dotted_var }} longer 
{% endfor %}

*****************
reference node
*****************
{% for IP_Prefix in Alloc_IP_Prefix_var%}
sh route {{ IP_Prefix }}
sh route longer {{ IP_Prefix }}/{{ mask_var }}
sh bgp {{ IP_Prefix }}
{% endfor %}


*****************
{%- for IP_Prefix in Alloc_IP_Prefix_var%}
{%- set IP_split = IP_Prefix.split('.')[0:3] %}
{%- set a = IP_split[0] | int %}
{%- set b = IP_split[1] | int %}
{%- set c = IP_split[2] | int %}
{%- set combine_1 = [a,b,c,1] %}
tracert {{ combine_1 | join('.') }}
{%- endfor %}
*****************



{% endif %}

• When I define the template, I’d like to set the variables from the python file to be of different name when I work with the jinja2 template
• i = 0, I set i as 0 as an initial value, I usually set i as a 0 value to avoid any issues further with the code as I use i in for loops that utilize ranges.
• mask_var indicates the prefix length, dotted_var would be used as a notation when we generate the commands

================================Implementation================================================
{# ----------start of recovery---------- #}
{% if method == 'transfer' or method == 'recover' or method == 'swap' -%}
-----------------------------Removing IP Local pool-----------------------------

***********************
{{ source_device_var }}
**********************

conf t
{% for IP_Prefix in Recov_IP_Prefix_var -%}
{% set command ='no ip local pool DHCP_NAME ' -%}
{% with y = command -%}
{% for i in range(0, 255) if i<stop -%}
{% set IP_split = IP_Prefix.split('.')[0:3] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set combine_1 = [a,b,c+i,1] -%}
{% set combine_254 = [a,b,c+i,254] -%}
{{ y }}{{ combine_1 | join('.') }} {{ combine_254 | join('.') }}
{% set i = i+1 -%}
{% endfor -%}
{% endwith -%}
{% endfor -%}
exit

• start of implementation part of change, {# insert text here #} stands for a comment line in Jinja2 template.
• method also varies the generated set of configurations depending on the use case I’m trying to achieve in the change.
• {{ source_device_var }} — this notation calls the variable which came from src_node value from the python file.
• this section is an example on how to utilize jinja2 logic. Goal is to generate a negation command where I’ll delete the line that states the DHCP IP range. Put prefix IP (a.b.c.d) as an argument, then generate a line where it states the a.b.c.1 and a.b.c.254. I used split to generate a list where it contains the 1st to 4th octet as an individual value, set them as an integer then assign to a variable. combine_1 and combine_254 contains a list which consists of the 1st 3 octets as integer, then 4th value of 1 and 254 respectively. Then I used join(‘.’) in order to generate an IP address composed of the 1st 3 octets then ended with .1 or .254. See example below:

variable file

python script output

• we have 2 prefixes in the Recov_IP_block. Taking a look at 10.10.10.0 and 11.11.11.0, this generated a command that erases the DHCP pool that specifies range from 10.10.10.1 to 10.10.10.254. The same goes for 11.11.11.0
• the value of this config generation really shows when we have to move multiple /24 prefixes. Human error, at least on the syntax of the generated config for execution will be avoided.
• This is an example for /24, we‘ll generate one for /22 which is composed of multiple lines.

variable file

python script output

• now that we got a mask of /22, this means that we will generate a config that’ll erase all DHCP config lines under that range. Going back to the Jinja2 template, I used the c+i in {% set combine_1 = [a,b,c+i,1] -%}, initially the i value is 0 from range. But at the end of the loop, it increments by 1 due to {% set i = i+1 -%}. On the next iteration, from 10.10.10.1, it will be 10.10.11.1— this is 1 way that we could generate incremented IP addresses octet as if the function is subnetting. /22 generated 4 iterations since my variable {% set stop = 4 -%} when {% elif mask_var == 22 %}.
• this is the same in /21 as the example shows below, as if it’s subnetting /21 into x8 /24s — and it does that for x2 /21 prefixes indicated in the variables.

variable file

python script output

moving forward,

----------------------Removing network statement, static route, and Prefix_list-----------------------------

***********************
{{ source_device_var }}
**********************

conf t
router bgp 1234
 address-family ipv4 
{%- for IP_Prefix in Recov_IP_Prefix_var %}
  no network {{ IP_Prefix }} mask {{ dotted_var }} route-map ROUTE_MAP_NAME
{%- endfor %}  
  exit
 exit
{% for IP_Prefix in Recov_IP_Prefix_var-%}
no ip route {{ IP_Prefix }} {{ dotted_var }} Null0
{% set IP_split = IP_Prefix.split('.')[0:4] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set d = IP_split[3] | int -%}
{% set combine_seq = [a,b,c] -%}
no ip prefix-list PREFIX_LIST_NAME seq {{ combine_seq | join() }} permit {{ IP_Prefix }}/{{ mask_var }} le 32
{% endfor -%}
exit
wr


{% endif -%}

{# ----------end of recovery---------- #}

{# ----------start of allocation---------- #}

{%- if method == 'transfer' or method == 'allocate' or method == 'swap' -%}

--------------Configuring Static Route, Prefix-list, network statement, and DHCP-----------------------

***********************
{{ dest_device_var }}
**********************


conf t
{% for IP_Prefix in Alloc_IP_Prefix_var -%}
ip route {{ IP_Prefix }} {{ dotted_var }} Null0
{% set IP_split = IP_Prefix.split('.')[0:4] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set d = IP_split[3] | int -%}
{% set combine_seq = [a,b,c] -%}
ip prefix-list PREFIX_LIST_NAME seq {{ combine_seq | join() }} permit {{ IP_Prefix }}/{{ mask_var }} le 32
{% endfor -%}
router bgp 1234
 address-family ipv4
{%- for IP_Prefix in Alloc_IP_Prefix_var %}
  network {{ IP_Prefix }} mask {{ dotted_var }} route-map ROUTE_MAP_NAME
{%- endfor %}
  exit
 exit
{% for IP_Prefix in Alloc_IP_Prefix_var -%}
{% set command ='ip local pool DHCP_NAME ' -%}
{% with y =command -%}
{% for i in range(0, 255) if i<stop -%}
{% set IP_split = IP_Prefix.split('.')[0:3] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set combine_1 = [a,b,c+i,1] -%}
{% set combine_254 = [a,b,c+i,254] -%}
{{ y }}{{ combine_1 | join('.') }} {{ combine_254 | join('.') }}
{% set i = i+1 -%}
{% endfor -%}
{% endwith -%}
{% endfor -%}
exit
wr








{% endif -%}


{# ----------end of allocation---------- #}

• this section would generate a configuration that will delete the existing routing configuration from the specified source device then allocate those IP addresses to a new node. This activity is pretty common on ISPs but I won’t go into much detail about the use case. This automated config generation is convenient when you’re moving multiple IP addresses, to be advertised from node A -> node B.
• for demonstration’s sake, I’ll only go through method = ‘transfer’ to show a generated config that would transfer a prefix advertisement from Node A -> B

variable file

python script output pt. 1

python script output pt. 2

• to sum it up, we have specified x2 /21 prefixes in our Recov_IP_block (prefixes to be recovered from src_node, then allocate the same prefix as specified in Alloc_IP_block to be configured in dst_node.
• this would remove DHCP config, BGP network statement, static hold down route, and the prefix-list for those prefixes indicated — from a source node.
• then would generate a configuration script to configure those DHCP, BGP network statement, static hold down route, and prefix-list to the destination node.

The rest of the template would generate the postchecks commands. See the portion of the output below:

################################################POSTCHECKS################################################

{% if method == 'recover' -%}
*********************
{{ source_device_var }}
*********************

{% for IP_Prefix in Recov_IP_Prefix_var -%}
{% for command in check_commands_var -%}
{% for i in range(0, 255) if i<stop -%}
{% set IP_split = IP_Prefix.split('.')[0:3] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set combine = [a,b,c+i] -%}
{{ command }}{{ combine | join('.') }}
{% endfor -%}
{% endfor -%}
{% endfor -%}

{% for IP_Prefix in Recov_IP_Prefix_var %}
sh ip route {{ IP_Prefix }} {{ dotted_var }}
sh ip route {{ IP_Prefix }} {{ dotted_var }} longer 
{%- endfor %}


*****************
Reference node 
*****************
{% for IP_Prefix in Recov_IP_Prefix_var %}
sh route {{ IP_Prefix }}
sh route longer {{ IP_Prefix }}/{{ mask_var }}
sh bgp {{ IP_Prefix }}
{% endfor %}


*****************
{%- for IP_Prefix in Recov_IP_Prefix_var%}
{%- set IP_split = IP_Prefix.split('.')[0:3] %}
{%- set a = IP_split[0] | int %}
{%- set b = IP_split[1] | int %}
{%- set c = IP_split[2] | int %}
{%- set combine_1 = [a,b,c,1] %}
tracert {{ combine_1 | join('.') }}
{%- endfor %}
*****************



{% endif %}


{% if method == 'allocate' or method == 'transfer' or method == 'swap' -%}
*********************
{{ dest_device_var }}
*********************

{% for IP_Prefix in Alloc_IP_Prefix_var -%}
{% for command in check_commands_var -%}
{% for i in range(0, 255) if i<stop -%}
{% set IP_split = IP_Prefix.split('.')[0:3] -%}
{% set a = IP_split[0] | int -%}
{% set b = IP_split[1] | int -%}
{% set c = IP_split[2] | int -%}
{% set combine = [a,b,c+i] -%}
{{ command }}{{ combine | join('.') }}
{% endfor -%}
{% endfor -%}
{% endfor %}

{% for IP_Prefix in Recov_IP_Prefix_var %}
sh ip route {{ IP_Prefix }} {{ dotted_var }}
sh ip route {{ IP_Prefix }} {{ dotted_var }} longer 
{% endfor %}

*****************
reference node
*****************
{% for IP_Prefix in Alloc_IP_Prefix_var%}
sh route {{ IP_Prefix }}
sh route longer {{ IP_Prefix }}/{{ mask_var }}
sh bgp {{ IP_Prefix }}
{% endfor %}


*****************
{%- for IP_Prefix in Alloc_IP_Prefix_var%}
{%- set IP_split = IP_Prefix.split('.')[0:3] %}
{%- set a = IP_split[0] | int %}
{%- set b = IP_split[1] | int %}
{%- set c = IP_split[2] | int %}
{%- set combine_1 = [a,b,c,1] %}
tracert {{ combine_1 | join('.') }}
{%- endfor %}
*****************



{% endif %}

variable file

python script output

Jinja2 is a good way to generate a templated configuration that follows various situation, logic, and variables. Of course, there are other methods to template commands, whether it would be for onboarding a device or for day-to-day network operations. There are also other proprietary SDN products that are capable of doing this but the value of using Jinja2 and python is that it’s more flexible in your specific use case plus it’s free.

all information written here are publicly available and gathered from the resources indicated below. Use case indicated in this writing is from the personal project of the author (John Abat) but components (such as BGP, static route, DHCP, and prefix-list) are common in day-to-day ISP/Network Operations.

https://pyneng.readthedocs.io/en/latest/book/20_jinja2/example.html

https://jinja.palletsprojects.com/en/3.1.x/

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-16/irg-xe-16-book/configuring-a-basic-bgp-network.html

https://github.com/dominicabat/Configuration_Generation_Jinja2