Python3 Regex Help.

[Stealthychu]

I currently have a script setup to ssh into a network device and extract parameters into a mysql database.

My data base displays the values: Hostname, Model, Image, IOSVersion, CPUModel, CPUSpeed, MACAddress and IPAddress.

Currently the script works to the point where it connects through ssh successfully and extracts the first six values but i believe my code relating to extracting the MAC Address and IP Address are wrong but am un-sure how to fix this.


In mysql table ti displays 'unknown' for both values so i assume i just have the re.search or re in general values wrong but am unsure how to fix it.
Script:

#Extract parameters ONLY from network device

#Send to MYSQL database NetMon

# cpu_speed and cpu_model do not work on 2811 routers in N523

import pymysql as mdb
import pymysql.cursors
#import MySQLdb as mdb
import paramiko
import os.path
import subprocess
import datetime
import time
import sys
import re


#MySQL Database Details

sql_host = "localhost"
sql_username = "admin"
sql_password = "python"
sql_database = "NetMon"



# Connect to MYSQL
def sql_connection(command, values):

    sql_connection = mdb.connect(sql_host,sql_username,sql_password,sql_database)

    cursor = sql_connection.cursor()
    cursor.execute("use NetMon")
    cursor.execute(command, values)

    sql_connection.commit()

#Open ssh connection

ip_address = "192.168.2.101"
username = "admin01"
password = "cisco12345"

ssh_client = paramiko.SSHClient()

ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(hostname=ip_address,username=username,password=password)

remote_connection = ssh_client.invoke_shell()
remote_connection.send(b"terminal length 0\n")
time.sleep(1)


#Reading commands

selected_cisco_commands = b'''show version | include (, Version|uptime is|bytes of memory|Hz)&\
                                  show inventory&\
                                  show interfaces | include bia&\
                                  show processes cpu | include CPU utilization&\
                                  show memory statistics&\
                                  show ip int brief | include (Ethernet|Serial)&\
                                  show cdp neighbors detail | include Device ID&\
                                  show ip protocols | include Routing Protocol'''

#Splitting commands using the "&" = and
command_list = selected_cisco_commands.split(b"&")


for each_line in command_list:
    remote_connection.send(each_line + b'\n')
    time.sleep(3)



#Checking command output for IOS syntax errors
output = remote_connection.recv(65535)

if re.search(b"% Invalid input detected at", output):
    print ("* There was at least one IOS syntax error on device %s")

else:
    print ("* All parameters were extracted from device %s")
print (output.decode('ascii') + "\n")


#Extracting Device Parameters


dev_hostname = re.search(b"(.+) uptime is", output)
hostname = dev_hostname.group(1)
print (hostname.decode('ascii'))



dev_model = re.search(b"(.+?) (.+?) (.+) bytes of memory", output)
model = dev_model.group(2)
print (model.decode('ascii'))

dev_image_name = re.search(b" \((.+)\), Version", output)
image_name = dev_image_name.group(1)
print (image_name.decode('ascii'))

dev_os = re.search(b"\), Version (.+),", output)
os = dev_os.group(1)
print (os.decode('ascii'))

cpu_model = ""
if re.search(b".isco (.+?) \((.+)\) processor(.+)\n", output) == None:
    cpu_model = "unknown"
else:
    cpu_model = re.search(b".isco (.+?) \((.+)\) processor(.+)\n", output).group(2)
print (cpu_model)

cpu_speed = ""
if re.search(b"(.+?)at (.+?)MHz(.+)\n", output) == None:
    cpu_speed = "unknown"
else:
    cpu_speed = re.search(b"(.+?)at (.+?)MHz(.+)\n", output).group(2)
print (cpu_speed)



mac_address = ""
if re.search(b"([0-9a-fA-F]:?){12}\n", output) == None:
    mac_address = "unknown"
else:
    mac_address = re.search(b"([0-9a-fA-F]:?){12}\n", output)
print (mac_address)


ip_address = ""
if re.search(b"[0-9]+(?:\.[0-9]+){3}\n", output) == None:
    ip_address = "unknown"
else:
    ip_address = re.search(b"[0-9]+(?:\.[0-9]+){3}\n", output)
print (ip_address)

#Insert/Update if exists all network devices data into the MySQL database table NetworkDevices. To access login to sql
sql_connection("REPLACE INTO NetworkDevices(Hostname,Model,Image,IOSVersion,CPUModel,CPUSpeed,MACAddress,IPAddress) VALUES(%s, %s, %s, %s, %s, %s, %s, %s)", (hostname, model, image_name, os, cpu_model, cpu_speed, mac_address, ip_address))

poll_timestamp = datetime.datetime.now()


#Closing the SSH connection
ssh_client.close()

[Gribouillis]

Do you have an example of a real "output" byte string from which these data are extracted?

[Stealthychu]

what do you mean by that sorry?

---

do you mean like the results of entering the commands in the routing device?

[Gribouillis]

I mean that you have many expressions like re.search(b"....", output). I want to know a typical contents of the output variable.

[Stealthychu]

The output variable is a combination of all the commands entered into the network device.
The output is the results given by the router when these commands are entered:
'show inventory'
'show interfaces'
'show processes cpu'
'show memory statistics'
'show ip int brief'
'show cdp neighbours'
'show ip protocols'

in the two i am having issues with they would be using the part of the output which follows:

The output that displays the IP Addresses I want to extract is: 'show ip interface brief' which gives the following output:



The output that displays the MAC Address I want to extract is: 'show interfaces'

[Gribouillis]

It seems that there is no newline in the string after the ip address, it means that you should probably remove the \n in the regular expression, so use re.search(br"[0-9]+(?:\.[0-9]+){3}", output). Also a good rule is to always use raw strings when writing literal regular expressions in python code, so use br"..." instead of b"..." when you write regular expressions.

I cannot see the mac address in your second picture, where is it? Or perhaps it is the c201.1110.0000 which is one possible form of encoding a mac address as wikipedia says, that is to say three groups of 4 hexadecimal digits, which could be written as the regex br"[0-9a-fA-F]{4}(?:\.[0-9a-fA-F]{4}){2}" but not all mac addresses have this form.

[Stealthychu]

Thanks, for your help so far.

I believe the address I am trying to extract is this one (circled in blue).

[Gribouillis]

I believe the address I am trying to extract is this one

See my edits in the previous reply.

[Stealthychu]

I attempted to change the code for the IP address extraction as you suggested to this.

ip_address = ""
if re.search(br"[0-9]+(?:\.[0-9]+){3}", output) == None:
    ip_address = "unknown"
else:
    ip_address = re.search(br"[0-9]+(?:\.[0-9]+){3}", output)
print (ip_address)

But i am no met with this error when i attempt to run it and the sql table still shows the IP Address value as unknown.

---

I changed it as you suggested with the MAC address as well and it seem i go the same error except with an extra line stating it found a match for the correct MAC address but gave the same errors as before after that.

[Gribouillis]

Then you need to learn python's re module, because re.search() returns a match object and you want a bytes string, so use

ip_address = re.search(br"[0-9]+(?:\.[0-9]+){3}", output).group(0)

The same applies to the rest of the code.