iSCSI vs NFS vs SMB

Block vs file at a glance

iSCSI: remote block storage

iSCSI puts SCSI commands inside TCP/IP packets. The server (target) presents one or more LUNs (logical units, essentially virtual disks). The client (initiator) connects to a LUN and uses it as if it were a locally-attached disk — partitions it, formats it with any filesystem, mounts it.

Strengths:

• Lowest protocol overhead for single-client throughput.
• Filesystem flexibility — use whatever filesystem the OS supports.
• Block-level features (snapshots, thin provisioning) on the storage side.
• Standard for virtualization platforms (VMware uses iSCSI or NFS; Hyper-V uses iSCSI or SMB).

Weaknesses:

• Only one initiator per LUN unless using a cluster-aware filesystem.
• Not appropriate for ad-hoc file sharing.
• Security model is per-LUN, not per-file.

NFS: Unix-native file sharing

NFS exposes a directory on the server to clients as a mountpoint. Clients see standard file and directory operations — open, read, write, close, chmod. The server handles the actual filesystem.

Strengths:

• Simple to operate; the protocol is well-understood.
• Stateless in NFSv3 (resilient to server restarts); NFSv4 added state for better semantics.
• Excellent for high-throughput sequential workloads.
• Cross-platform (clients on macOS, Linux, Windows, BSD).

Weaknesses:

• Default authentication (sys, based on UID/GID) is essentially trust-based. Strong auth requires Kerberos integration.
• Locking is per-NFS-server; doesn't extend across multiple NAS heads.
• Performance is sensitive to network latency; small-file workloads suffer.

SMB: Windows-native file sharing

SMB (formerly CIFS) is the file protocol on Windows networks. Like NFS, it presents files and directories from the server to clients. Unlike NFS, it has integrated authentication, native Windows ACL support, and a richer set of operations (opportunistic locks, change notifications).

Strengths:

• Built-in authentication (NTLM, Kerberos) integrated with Active Directory.
• SMB 3+ supports encryption, multichannel, and continuous availability.
• Windows ACL semantics preserved natively.
• Better small-file performance than NFS in many cases.

Weaknesses:

• Historically chatty protocol with significant overhead.
• SMB 1.x had security issues; modern deployments must disable it (WannaCry exploited SMB 1).
• Cross-platform performance varies — Linux SMB clients (cifs/Samba) lag native Windows.

Performance characteristics

Rough order for single-client raw throughput on identical hardware:

1. iSCSI — least overhead per byte.
2. NFSv3 over UDP for large sequential reads — minimal stateful chatter.
3. NFSv4 / SMB3 — more overhead due to richer semantics.

For small-file workloads (build systems, mail servers, source code), SMB often outperforms NFS because of its better metadata handling and oplocks. For large sequential reads (video, backups), all three can saturate the link if the underlying storage can keep up.

Encryption

• iSCSI — IPsec or Tunneled TLS. Not commonly enabled in trusted networks.
• NFS — Kerberos with privacy (krb5p) provides encryption. Without it, NFS is plaintext.
• SMB 3+ — native encryption per-share. Strongly recommended; enable by default.

For storage traffic crossing untrusted networks (cross-site replication, cloud-to-on-prem), encryption is mandatory regardless of protocol.

The virtualization angle

VM hypervisors typically use one of three storage models:

• iSCSI to a LUN containing VMFS (VMware) or similar cluster FS — block, multi-host capable via the cluster FS.
• NFS to a datastore — file, multi-host native, simpler ops.
• SMB 3 multichannel — Hyper-V uses SMB 3 directly for VHD storage.

For VMware: NFS and iSCSI are both common. iSCSI for raw performance, NFS for ops simplicity. For Hyper-V: SMB 3 is the modern default.

The home NAS angle

For a home or small office NAS:

• Linux/macOS clients — NFS or SMB. NFS is faster but requires shared UID/GID; SMB just works with usernames.
• Windows clients — SMB.
• Mixed — SMB. Cross-platform reach with one share.
• iSCSI — only if you have a specific block-storage need (Time Machine backup target on macOS, VM datastore on a hypervisor host).

Locking models

For multi-client concurrent access, the locking model matters:

• NFSv3 — separate NLM (Network Lock Manager) protocol; sometimes flaky.
• NFSv4 — locking integrated into the main protocol; cleaner.
• SMB — opportunistic locks (oplocks); the client caches and the server can revoke. Rich and well-tested but Windows-specific in some semantics.
• iSCSI — no locking at the protocol layer; must come from the filesystem on top (cluster FS or single-client).

Network requirements

All three protocols benefit from:

• 10GbE or better for high-throughput workloads. 1GbE saturates around 110-120 MB/s — limiting for modern NVMe-backed storage.
• Jumbo frames (MTU 9000) when both endpoints support it; reduces overhead.
• Dedicated storage VLAN to avoid contending with regular traffic.
• For iSCSI specifically: multipathing (MPIO) for redundancy and bandwidth aggregation.

Frequently Asked Questions

What is the difference between block and file storage?

Block storage exposes raw storage blocks (sectors); the client formats and manages a filesystem on top. iSCSI is the dominant block protocol — the server is essentially a remote disk. File storage exposes files and directories; the server runs the filesystem and the client uses standard file operations. NFS and SMB are file protocols. Block storage is typically lower-overhead and supports a single client per LUN; file storage handles multi-client access natively.

What is iSCSI?

Internet Small Computer Systems Interface — encapsulates SCSI commands inside TCP/IP so a client can access remote storage as if it were a local disk. The client (initiator) mounts an iSCSI target (LUN) and treats it as a block device — partitions it, formats it, mounts it. Used heavily for virtualization storage where each VM gets its own LUN.

What is the difference between NFS and SMB?

NFS is the Unix-native file protocol; SMB is the Windows-native file protocol. Both work cross-platform today, but each has its native ecosystem. NFS is typically simpler to operate, uses port 2049, and has weaker default security (relies on Kerberos for strong auth). SMB has built-in authentication, supports Windows ACLs natively, and is the default on Windows file shares.

Which protocol is fastest?

In single-client benchmarks, iSCSI is typically fastest because it has less protocol overhead — there's no filesystem semantics to coordinate, just block reads and writes. NFS and SMB add filesystem-level operations (open, close, lock, attribute lookups) that consume bandwidth and latency. For multi-client workloads, the picture changes — file protocols handle concurrent access; iSCSI requires a cluster-aware filesystem or a single client per LUN.

Can I use iSCSI for general file sharing?

No, not directly. iSCSI provides block access, not file sharing. Two clients connected to the same iSCSI LUN with non-cluster-aware filesystems will corrupt the filesystem because each thinks it owns the disk. iSCSI is for single-client access (one server, one LUN) or for clustered filesystems designed for shared block storage (VMFS, GFS2, OCFS2).