Extreme Networks – EXOS BGP MED

I came across a scenario where a user had two data centers in different locations connecting back to the same ISP via BGP. These two data centers would be advertising a unique /24 at each site. However, the user also wanted to advertise the other DC’s /24, but not in an active state for failover. Being that the user was connecting back to the same provider AS, I decided to test using the BGP MED (Multi Exit Discriminator) attribute to determine which /24 would be the preferred route from the provider end. The MED with the lowest value takes a higher priority.

We’re using Extreme Networks summit series switches, so I tested the configuration on exos 22.6 using my EXOS virtual lab. I made sure to apply a lower MED value to the /24 I wanted to prioritize at each primary site and also applied a higher MED value to the backup /24 at the opposite site. 

In summit series switches you start with a policy file that matches the network address used in the BGP network statement. You can then apply a med value to that match. EXOS uses vi when creating these policy files. Here are the commands:

exos-vm#vi med_add.pol

entry med_add {

If   {



Then   {

med set 100;




You can finish the policy file with a :wq

Apply the policy using the BGP network command

exos-vm#Configure bgp add network network-policy med_add

In my lab, I was using two different AS numbers representing each DC connected back to the same AS. Therefore I also had to use the “enable bgp always-compare-med” command on the simulated provider AS exos virtual switch as MED values are not compared between routes advertised from different autonomous systems by default.

Of course, your provider has to be willing to accept MED. If not, you could also try prepending your AS number to the AS_Path. This is another way to manipulate what route is less preferred. However, this method is not always supported as some providers ignore duplicate AS in the AS_path. The change for AS_Path is simple, just replace the med set 100; with As-path “2020“; in the policy file. This example is using AS path 2020 and should be applied to the BGP network statement that serves as the backup route at the opposite DC location.


Extending Layer 2

extended bridge

Layer 2 Bridging

If you make your way into the world of networking, you’re bound to come across a decision path on how you should handle network expansion. Should your default method always be to extend or stretch your layer 2 bridge domain? The root of the answer can be found when discussing the why. Let’s take a look at some of the use cases I’ve come across within enterprise network environments:

Device Requirement Device “A” needs to communicate with device “B” and those two devices are “required” to live on the same layer 2 broadcast domain. I haven’t come across any new devices or applications that fall into that spectrum, and it’s 2018. However, some enterprise organizations may still have legacy devices or poorly manufactured devices/applications with no foreseeable updates that may fall into this category.

Customer demand A customer you service in area “A” needs network services expanded to area “B.” They want their equipment to stay on the same subnet. Cough, cough, point of sales systems. I believe that modern POS systems can talk via IP across different subnets, but this can also be a possible use case that still comes up.

Data center disaster recovery or should I say “specific” DR models. I say “specific” because not all DC DR needs to be developed with an absolute layer 2 requirement extension model. Specific apps that are short-sighted will include layer 2 extension as a requirement. Someone insists that a VM pinned to a specific IP move from region “A” to region “B” and the IP needs to stay the same. What!?! Let’s think of better ways to do this, DNS, automate IP provisioning? However, this can still be a possible use case.

Ease of use Sometimes if you’re uncomfortable with routing protocols, it may seem easier to span a VLAN across the core of the network. Less IP provisioning, less ACL’s, potentially less firewall rules, and less management of those dreaded IP routing protocols. However, this is something we are in control of, so it’s OK to take time to research and learn what routing protocol would work best for your environment. Don’t let the lack of information drive your operation.

I can confirm that extending 100’s of VLAN’s through your core along with multiple instances of STP with a sprinkle of HSRP is NOT scalable. You will run into issues at some point. Others would say, “but my superior wants things done yesterday.” That’s another topic which may be worth blogging about in the future but hang in there.

You’re getting the point. There are some better Continue reading »

Why I’m so fascinated with white box switching

All the hype

White box switching seems to be all the networking hype.  For some in-depth research, check out this podcast from packet pushers about ATT making its move into white box switching. Cisco is also committed to offering a decoupled version of IOS-XR from Cisco hardware to enable running their NOS on OCP (open compute project) compliant hardware aka “white box switching.” Fascinating stuff, but what’s the big deal? Well, I’m going to try and make a comparison.

A Lego comparison

I’m a huge adult fan of Lego (AFOL). I remember dumping old tin popcorn bins with Legos all over my bedroom floor as a child. I’m more organized today, but I can’t help tearing down and building new creations. Now imagine you have an advanced Lego technic set put together. You have gears that move, hinges that open and close, wheels turn, etc. Now imagine all those connecting pieces glued together. A nightmare for those AFOL’s who want to rebuild something special.

white box switch lego

Picture that glued together Lego set as a networking switch or router. Sure you can plug and unplug a few items, configure features within the CLI, and even get some sweet stats via SNMP. However, your switch or router’s underlying code is static which you can’t change. You’re at the mercy of the vendors nicely glued together product. I’m not suggesting that’s necessarily a bad thing, but you get where I’m going. With white box switching, you finally get to be a bit more creative with your switch or router. You can unload the default network operating system and load up something completely different. You’ve just expanded your imagination beyond one vendor and their fixed code.

A modular future

Maybe we’ll start to see advanced hardware modularity for white box switching as well. You need more processing power; upgrade your CPU. You need more space for your NOS apps or massively large routing tables, then go ahead and add more RAM. Are you a Cisco or Cumulus fan, who cares, you choose what NOS to run. Now you’re building like an AFOL. The possibilities of customization that deliver high flexibility are endless.

Lego 16 port white box switch

Extreme Networks EXOS on Nutanix CE

EXOS in Nutanix CE

Now that I have my Nutanix CE lab setup, I wanted to get some of my virtual network operating systems installed within my home lab. One of the NOS’s I’ve been running is Extreme Networks virtual EXOS. My last EXOS-VM lived in Virtualbox and ESXi. Extreme Networks has a github page here with all the information you need to get started with running the VM within a Virtualbox or ESXi environment.

Issue and Solution with Nutanix

Following the EXOS installation guide using the downloadable iso and mimicking the Vmware/Virtualbox VM settings within Nutanix CE wouldn’t work. I kept receiving an issue with the disk not correctly detected while Continue reading »

Best of Breed

I recently heard the term “best of breed” used when discussing network vendor selection. I was surprised by this answer because you don’t hear it too often.  The more I thought about it, why not “best of breed” selection? My time as a network and infrastructure supervisor has taught me that a data center environment can be full of different compute and storage vendor products. Our SAN environment consists of Pure, Tegile, EMC, and even QNAP. Each product has its place.  Pure serves the VDI environment, Tegile/EMC host production, and QNAP serves as a target for our Veeam backups. The team has also categorized and carved out each platform into tiered offerings.

On the other hand, network vendor selection tends to be biased. Typically you’ll see one network vendor selected for the edge/access, distribution, and core. However, you will find a different wireless vendor from time to time.

Many reasons exist

A compilation of the most popular

  • We would like to interact with only one vendor for purchases and support.
  • ABC vendor only works well with a particular management tool.
  • I only know vendor ABC, and we don’t have time to learn something new.
  • Did you hear that vendor ABC had an issue with XYZ product, I don’t want those problems.
  • Everyone else uses vendor ABC.
  • No other vendor supports my VOIP feature set.
  • You can’t do XYZ well or at all with any other vendor product.

I will say that there are a few use cases that keep you tied Continue reading »

Extreme Networks – enabling a few things on b5/c5/k series

Extreme Networks Switching Commands

Some of my most visited posts seem to be on brocade switching configuration/commands, so I decided to put together our standard list of commands for some Extreme Networks switches we use. These commands can be used on the B5, C5, K series, 7100 series, and S Series Extreme Networks switches. These switches run the EOS network operating system. Extreme networks product line moving forward will be purely EXOS (ExtremeXOS operating system). Therefore the following commands will become legacy, but are still very useful to know since some of the EOS product line hasn’t reached EOL.  Some commands are self explanatory, but for other’s I added a short description. Continue reading »

Troubleshooting ARP/IGMP/Router CPU

The Issue

We recently starting having issues with a building reporting that icmp stopped responding on a distribution router and some access switches behind the router. Some routing interfaces would respond, but the management VLAN interface wouldn’t. Further troubleshooting showed that the CPU processes on the router comprised of two Extreme Networks 7100 series switching running OSPF climbed up to 80/100% utilization. The “show logging buffer” revealed massive amounts of host-dos ARP attack events. The first thought was that a possible infected machine was creating an ARP storm. Continue reading »

Cisco Virtual Internet Routing Lab – Up and Running…

I was finally able to fix the issue that I described having in my earlier Cisco VIRL article. My original bare metal box only had 3 NICs. VIRL requires that you have at least 5 NICs. If you don’t have 5 NIC’s, then you have to modify the /etc/virl.ini file with dummy interfaces. I did this earlier, but must have had a mistake in the config. I double checked the config and also ran the VIRL-rehost script that’s on the desktop when you login to VIRL. Running the script wasn’t in the VIRL doc steps, so I didn’t do this before. Running the script after modifying the virl.ini file with the dummy interfaces finally fixed my issue. The script modified the /etc/network/interfaces with the correct dummy interfaces. Here’s an example of what the script changed:

iface dummy1 inet static



post-up ip link set dummy1 promisc on

I setup an IOSv router and connected it to an L2 External (flat) network which connects back through one of my physical NICs. That connection then goes into a real cisco ws-c3560x 24 port switch. That switch is connected to my network and I assigned my PC another IP address on the 172.16.1.x network. I can now ssh into the IOSv router directly from my desktop.

virl flat network


Now that I have worked out all the bugs, I’m pretty impressed with the functionality that VIRL provides. Now I’m going to see how many routers I can throw at this box.

Ping you later,


Cisco VIRL setup

In my last post, I spoke about getting a cisco virl (virtual internet routing lab) server going. I started with a hyper-V installation, which wasn’t listed as being supported. I gave it a try anyways. What I came to find out is that hyper-V would not work with my setup because I couldn’t do nested virtualization. Cisco Virl runs KVM under the hood which needs native VT-d. I couldn’t get hyper-v to pass VT-d to the host, so that was a no go. I decided to wipe the drive and load the iso version of cisco virl directly on my box. After a few failed attempts, I finally got virl to run without giving an error when trying to license the software. You have to follow the virl install guide to the T. I accidentally didn’t put in the correct hostname that the guide said to use. That caused the initial installation of virl to give an error every time I tried to run the user workspace manager. After following the exact directions, I was able to get virl to load.

Now I’m running into one last error message:

state changed from BUILD to ERROR with message: No valid host was found. Exceeded max scheduling attempts 3 for instance

cisco virl

I’m digging into the logs in order to figure out what the issue is, I’m really close. Hopefully my next post will be an in depth review of cisco virl.

occupied, time to setup Cisco VIRL

I’ve been occupied with many other things going on lately especially with the holidays. Family, work, etc. I haven’t been able to sit and write, but I ran into something pretty cool that I wanted to share. Those of you who are always looking to expand your knowledge in networking, cisco just released VIRL or virtual internet routing lab. This is Cisco’s newest virtualization lab simulation tool. It reasonably priced as well.

I saw this as a perfect opportunity to build a homebrew cisco VIRL hyper-v server. Unfortunately hyper-v isn’t supported, but that didn’t stop me. I recently found a dell studio 435 with an i7-920 processor in the garbage down the street. The only issue it had was a dying power supply.

cisco VIRL home server

Once that was fixed, I started to install hyper-v 2012. To my dismay, I couldn’t get hyper-v loaded without the installation crashing. All settings looked to be good. Virtualization was enabled in the bios and all other recommended settings were set. I then realized that the bios version was pretty old and figured I try to do an update. Low and behold a bios firmware fixed the crashing issue. I slapped another broadcom pci-e nic and finished the install. Now I’m having issues getting the broadcom nic and the onboard nic to load correctly. After searching the Internet, I figured out how to manually add drivers in hyper-v core mode. I’m still running into some issues with NIC detection, so I think I may run esxi on another partition of the drive just to get VIRL running. Stay tuned….

You can now follow up on my cisco bare metal home build here.


1 2