NetApp Cluster-mode networking deep dive!

While administering NetApp cluster-mode, it's very important to understand networking and related concepts for successful troubleshooting in case of network failures. In this post I will be discussing the key network concepts related to cluster-mode.

Networking has gained special focus in cluster-mode with a goal to integrate fully qualified, high-performance switches into the sacle-out cluster and management networl

Key concepts:
==========
-Each controller have multiple physical network ports depending upon the controller model
-Physical network ports and logical interfaces (LIFs) are two different entities
-SAN environments use HBA ports as data ports



Roles:

Cluster ports: 
==========
-Defaults to two ports for each node
-These ports are reserved strictly for the intracluster traffic

Data ports:
========
-Defaults to two ports for each node
-These ports are used for client data access and for the cluster management LIF
-These can NICs or FC ports depending upon the NAS or FC SAN environments

Node-Management ports:
===================
-Defaults to one port for each node
-These ports are used by administrators to configure the node
-Can be connected through SSH or a web-browser

Intercluster ports:
=============
-Defaults to none for each node
-These are used to communicate with another cluster
-These can be failed over to data ports
-Data ONTAP cluster-mode has the ability to do intercluster (or cross-cluster) data-protection mirrors that is performed  over ports that have either the role of intercluster or data

To view available network ports:

net port show

To view available fiber-channel ports:

network fcp adapter show

Interface group (ifgrp):
=================
-Same as a virtual interface (vif) and port trunk
-It combines two or more physical ports and presents them as single logical port
-ifgrp provides redundancy and more bandwidth

Logical Interface (LIF):
=================
-IP addresses are associated with network ports; many data IP addresses can be assigned to a single network port
-A node can have a maximum of 128 LIFs
-Contains one node management LIF per node that cannot be failed over or migrated
-Contains one cluster management LIF per cluster that can be failed over or migrated throughout the cluster
-Contains two cluster LIFs per node that can be failed over or migrated within its node
-Contains multiple data LIFs per data port to face the client allowing NFS, CIFS, iSCSI and FCoE that can be migrated or failed over throughout the cluster
-FC LIFs use WWPNs

To view available logical interface ports:

network interface show

LIF roles compatibility with port roles:
=============================
Data----------------->Data
Cluster-------------->Cluster
Intercluster--------->Intercluster, Data
Node-management------>Node-management, Data
Cluster-management--->Node-management, Data

LIFs routing:
=========
-Configure routing groups and static routes to control LIFs in a Vserver to use network for outbound traffic
-A routing table contains each LIF association with one routing group to use the routes of that group
-Multiple LIFs can share a routing group
-A static route can be defined between a LIF and a specific destination IP address

To view the available routing-groups:

network routing-groups show

To view the route information:

network routing-groups route show

LIF failover - Automatic failover during outage or reboot
LIF migrate - For manual failover scenario
LIF revert - Manually or automatically sending LIF back to its home

Note: LIF failover is controlled by the failover policies and failover rules

LIF failover policies:
==============
nextavail: The LIF moves to the next available working port in a round-robin fashion
priority: The LIF moves to the port or node as specified
disabled: LIF does not failover

To view the LIF failover policy:

network interface show

->look for failover policy

To view the summary of failover ports:

network interface failover show

Optimized and Nonoptiomized paths:
===========================
-Optimized paths are the primary or favored paths that are active between the initiator and target that have same latency level
-NonOptimized paths are the secondary or unfavored paths that are inactive between the initiator and target that have different latency level


For us to provide multipath access there are two available choices:

Symmetric: In this all paths are favored or optimized
Asymmetric: In this only certain paths are favored or optimized

Choosing optimized paths can be done manually (you don't want to do it!) or automatically by using ALUA (asymmetric logical unit access)

ALUA:
======
-Also known as Target Port Group Support (TPGS) in the SCSI primary commands (SPC) standard
-ALUA can operate in the following possible states:

a. Active/optimized
b. Active/nonoptimized
c. Standby
d. Unavailable