Upgrades, upgrades and again upgrades, nowadays it’s all about software upgrades. They are always announced as an easy and quick process, but is this true? In case of NSX Advanced Load Balancer, I can confirm. The upgrade process itself is very easy and straightforward. But nevertheless upgrades always needs some preparation. Therefore here some short upgrade checklist you can use if you are planning to upgrade to a newer release of NSX Advanced Load Balancer.
Upgrade checklist
1. Define the new version to upgrade to.
An upgrade always starts with a reason. This reason can be very broad: It can go from bug fixes, new features to compatibility with other product versions and many more…
It’s always good to have a good overview of what the goal of the upgrade is. This goal should be clear for every involved stakeholder. Based on the reason, a suitable version can be chosen. In most cases it will be one of the latest. To define a version it would be good to read the release notes of different versions. I will give you a good description about what bugs are fixed, which new features are present, or even what existing bug are still present.
Release notes: https://avinetworks.com/docs/latest/release-notes/
2. Verify the system requirements, compatibilities and upgrade path.
Based on the reason for the upgrade, a version was chosen. Now you should be sure that this version isn’t breaking anything else in your environment.
- For that reason it’s good to check the VMware Interoperability Matrix: https://interopmatrix.vmware.com/Interoperability?col=789,&row=0,. By consulting this matrix you can verify if different VMware products are compatible or not. This might lead to upgrades of different VMware products as well, depending what priorities you defined. Do you really need the new version?
- Furthermore you should check the system requirements for that version. it could be that some nodes needs additional scaling, licenses etc: https://avinetworks.com/docs/latest/system-requirements-ecosystem/
- Last but not least, verify what the supported upgrade path is for the current version towards to the new version. It might be that no direct path is supported. In that case multiple upgrades needs to be performed to reach the final (chosen) version. You can find these often in the release notes: https://avinetworks.com/docs/latest/release-notes/
3. Define if downtime is allowed during the upgrade.
Management/Control plane
During an upgrade of the NSX Advanced Load Balancer it’s currently not possible to avoid management plane down time as all controller in the cluster initiate the upgrade simultaneously. Just be aware of this downtime when planning the upgrade.
Data plane
Downtime of the data plane depends on the current setup and configuration. The chosen Virtual Service HA and placement options will define if your service will have downtime or not during the upgrade of the Service Engines. If you have no idea what I’m talking about, I would like to invite you to read my previous post:
4. Define how the upgrade will be performed: UI, CLI or REST API.
Depending on the source version, there are different possibilities on how to perform the upgrade. Each way has its advantages and disadvantages. Let’s find them out:
UI
<I love clicking around webpages>
- Only starting with Avi Vantage release 20.1.1
- Upgrading only Avi Controller
- Upgrading Avi Controller and Avi Service Engine groups
- Upgrading only Service Engine Group
- Easy and automated rollback in case of failure
- …
CLI
<I’m used and love to use commands>
- Better control of the upgrade
- Checks are directly integrated into the normal work-flow.
- Login with ssh and switch to CLI mode by executing “shell” command
- Easy and automated rollback in case of failure
- …
REST API
<I’m more like an automating guy>
- Upgrade can be automated with your own tools.
- Pre-checks are performed manually by adding
/preview/
at the end of the APIs URI - SE Upgrades have additional parameters for faster or safer upgrade: Disruptive – Suspend-on-failure
- …
Official upgrade documentation:
- UI: https://avinetworks.com/docs/latest/flexible-upgrades-using-avi-ui/
- CLI/REST API: https://avinetworks.com/docs/latest/flexible-upgrades/
5. Find the leader node
When performing the upgrade via CLI or REST API, you have to upload the images on and run the upgrade from the leader node.
You can find the leader node in the UI. Login in to the UI -> Administration -> Controller -> Nodes
6. Be sure backup is working fine
Always good to have backups before starting a major change or upgrade. Be sure that your backups are stored in a safe and remote location. Until now NSX ALB is only supporting SCP as protocol to store the backup to a remote location. (S)FTP is not (yet) supported.
In case you are not able to store the backup remotely, you can download the JSON-file locally to your compute by clicking the local file of a certain timestamp.
7. Verify licenses
As Service Engines / Service Cores are licensed, you don’t have to worry about the amount of licenses when upgrading. You should only think about the amount of licenses in case you change Virtual Service HA and placement options.
Since version 20.x, they have changed the licenses, but here again, you don’t need to worry. Your current licenses will be converted to equivalent Service Core-based licenses when an Avi controller is upgraded. (
You can read more about licenses: https://avinetworks.com/docs/latest/license-management-starting-with-release-20.1.1/
8. Better be safe then sorry: collect logs before and after
Quick last tip: Generate a Tech support bundle before and after the upgrade in case anything goes wrong. VMware support will be happy and grateful.
Tech support logs can also be collected via UI, CLI or REST API. Via the UI, you login and navigate to Administration -> System -> Tech Support.
I’m my opinion it will be good to get started with debug logs.
More info can be found here: https://avinetworks.com/docs/latest/collecting-tech-support-logs/
Summary of the upgrade checklist
- Define the new version to upgrade to.
- Verify the system requirements, compatibilities and upgrade path.
- Define if downtime is allowed during the upgrade.
- Depending on the source version, define how upgrade will be performed: UI, CLI or REST API.
- Find the leader node
- Be sure backup is working fine and exported to a remote location (or downloaded locally on your computer before starting the upgrade)
- Verify licenses
- Better be safe then sorry: collect logs
I am convinced about the fact this list is never 100% complete. Please take this checklist as an advice and not as an official guideline. Always check the official documentation and guides as products are changing over versions and time. If you have comments or remarks, feel free to contact me over socials.