How Does iptables Actually Work? — The Netfilter Hook Array and Packet Filtering

Every Linux administrator has used iptables to configure firewall rules. But what happens inside the kernel when a packet arrives? Most imagine a chain of linked lists — each rule checked sequentially until one matches. This is wrong.

Modern Netfilter uses compact arrays of hook functions, not linked lists. This seemingly small change has profound performance implications: arrays have better cache locality, no pointer chasing, and can be traversed with simple index increments. On a system with hundreds of iptables rules, this can mean the difference between 1M packets/sec and 500K packets/sec.

This article walks through the Netfilter source in net/netfilter/core.c to explain the five hook points, the nf_hook_entries compact array structure, and how nf_hook_slow() traverses the hooks.

Key Takeaways

• Netfilter has 5 hook points: PRE_ROUTING, LOCAL_IN, FORWARD, LOCAL_OUT, POST_ROUTING
• Hook functions are stored in compact arrays, not linked lists
• nf_hook_slow() traverses the array in priority order
• Return values: ACCEPT, DROP, STOLEN, QUEUE, REPEAT
• Conntrack hooks at PRE_ROUTING and LOCAL_OUT for connection tracking
• Per-netns hook arrays enable network namespace isolation

The Myth: "iptables is a Chain of Linked Lists"

The mental model: iptables rules form a linked list → each packet traverses the list → first matching rule wins. This is wrong.

What actually happens:

  Packet arrives → ip_rcv() → NF_HOOK(NF_INET_PRE_ROUTING)
       │
       ▼
  ┌─────────────────────────────────────────────────────────────────────┐
  │ nf_hook_entries (compact array, NOT linked list)                    │
  │                                                                     │
  │  ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐    │
  │  │hook[0]  │ │hook[1]  │ │hook[2]  │ │hook[3]  │ │hook[4]  │    │
  │  │prio=0   │ │prio=0   │ │prio=100 │ │prio=200 │ │prio=300 │    │
  │  │filter   │ │mangle   │ │nat      │ │filter   │ │mangle   │    │
  │  └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘    │
  │       ↑           ↑           ↑           ↑           ↑            │
  │       └───────────┴───────────┴───────────┴───────────┘            │
  │                   Array traversal (sequential memory)              │
  └─────────────────────────────────────────────────────────────────────┘

The Five Netfilter Hook Points

  Packet from network
       │
       ▼
  ┌─────────────────────────────────────────────────────────────────────┐
  │ NF_INET_PRE_ROUTING                                                 │
  │ • After ip_rcv(), before routing decision                          │
  │ • Used for: DNAT, conntrack, incoming packet mangling               │
  └─────────────────────────────────────────────────────────────────────┘
       │
       ▼
  ┌─────────────────────────────────────────────────────────────────────┐
  │ Routing Decision (ip_route_input)                                   │
  │ • Local delivery or forward?                                        │
  └─────────────────────────────────────────────────────────────────────┘
       │                              │
       ▼ (local)                      ▼ (forward)
  ┌─────────────────────┐    ┌─────────────────────────────────────────┐
  │ NF_INET_LOCAL_IN    │    │ NF_INET_FORWARD                         │
  │ • After routing,    │    │ • For packets being forwarded           │
  │   before protocol   │    │ • Used for: filter, mangle              │
  │   handler           │    │                                         │
  └─────────────────────┘    └─────────────────────────────────────────┘
       │                              │
       ▼                              ▼
  ┌─────────────────────┐    ┌─────────────────────────────────────────┐
  │ Protocol handler    │    │ NF_INET_POST_ROUTING                    │
  │ (tcp_v4_rcv)        │    │ • After routing, before transmission    │
  └─────────────────────┘    │ • Used for: SNAT, outgoing mangle       │
       │                     └─────────────────────────────────────────┘
       ▼                              │
  ┌─────────────────────┐             │
  │ NF_INET_LOCAL_OUT   │             │
  │ • From local process│             │
  │ • Used for: filter, │             │
  │   mangle, conntrack │             │
  └─────────────────────┘             │
       │                              │
       ▼                              ▼
  ┌─────────────────────────────────────────────────────────────────────┐
  │ NF_INET_POST_ROUTING                                                │
  │ • After routing decision, before dev_queue_xmit()                   │
  │ • Used for: SNAT, outgoing packet mangling                          │
  └─────────────────────────────────────────────────────────────────────┘
       │
       ▼
  Packet to NIC

nf_hook_entries: The Compact Array

// include/linux/netfilter.h
struct nf_hook_entries {
    u16                         num_hook_entries;  // Number of hooks
    struct nf_hook_entry        hooks[];           // Flexible array (compact!)
};
 
struct nf_hook_entry {
    nf_hookfn                   *hook;             // The hook function
    void                        *priv;             // Private data
};

Why Arrays Beat Linked Lists

nf_hook_slow(): The Traversal Engine

// net/netfilter/core.c — nf_hook_slow()
int nf_hook_slow(struct sk_buff *skb, struct nf_hook_state *state,
                 const struct nf_hook_entries *e, unsigned int s)
{
    unsigned int verdict;
    int i;
 
    // Traverse hooks in array order (sorted by priority)
    for (i = s; i < e->num_hook_entries; i++) {
        verdict = nf_hook_entry_hookfn(&e->hooks[i], skb, state);
        if (verdict != NF_ACCEPT) {
            if (verdict == NF_DROP)
                kfree_skb(skb);
            return verdict;
        }
    }
 
    return NF_ACCEPT;  // Default: accept if no hook drops
}

Return Values

NF_HOOK(): The Macro

// include/linux/netfilter.h
#define NF_HOOK(pf, hook, skb, indev, outdev, okfn)                     \
    NF_HOOK_THRESH(pf, hook, skb, indev, outdev, okfn, INT_MIN)
 
#define NF_HOOK_THRESH(pf, hook, skb, indev, outdev, okfn, thresh)      \
    nf_hook_thresh(pf, hook, skb, indev, outdev, okfn, thresh)
 
static inline int nf_hook_thresh(unsigned int pf, unsigned int hook,
                                  struct sk_buff *skb, struct net_device *indev,
                                  struct net_device *outdev, int (*okfn)(struct sk_buff *),
                                  int thresh)
{
    const struct nf_hook_entries *entry = rcu_dereference(net->nf.hooks[pf][hook]);
    if (entry && entry->num_hook_entries > 0)
        return nf_hook_slow(skb, &state, entry, 0);
    return okfn(skb);  // No hooks → continue normal processing
}

Conntrack: Connection Tracking

// net/netfilter/nf_conntrack_core.c
unsigned int nf_conntrack_in(struct sk_buff *skb, unsigned int hooknum)
{
    struct nf_conn *ct;
    enum ip_conntrack_info ctinfo;
 
    // Look up connection in hash table
    ct = nf_ct_get(skb, &ctinfo);
    if (ct) {
        // Existing connection — update state
        if (!nf_ct_is_confirmed(ct))
            ct = nf_ct_confirm(ct, skb);
        return NF_ACCEPT;
    }
 
    // New connection — create tracking entry
    ct = resolve_normal_ct(skb, hooknum, &ctinfo);
    if (!ct)
        return NF_DROP;
 
    return nf_conntrack_confirm(ct, skb);
}

Conntrack hooks at NF_INET_PRE_ROUTING and NF_INET_LOCAL_OUT to track all connections.

Deep Detail: Per-netns Hook Arrays

Each network namespace has its own hook arrays:

// net/netfilter/core.c
struct net {
    struct nf_hook_entries __rcu *nf_hooks[NF_MAX_HOOKS][NF_MAX_HOOKS];
    // ...
};

This means:

• Container A can have different iptables rules than Container B
• Rules in one namespace don't affect another
• The nf_hook_thresh() macro reads from the current netns's array

How to Observe Netfilter

Using bpftrace

#!/usr/bin/env bpftrace
// trace_netfilter.bt
 
kprobe:nf_hook_slow
{
    @hooks[comm] = count();
}
 
kprobe:nf_conntrack_in
{
    @conntrack[comm] = count();
}
 
tracepoint:net:net_dev_queue
{
    @tx[comm] = count();
}

Using /proc

// iptables rules
iptables -L -n -v
iptables -t nat -L -n -v
 
// Conntrack table
cat /proc/net/nf_conntrack
 
// Conntrack count
cat /proc/sys/net/netfilter/nf_conntrack_count

Frequently Asked Questions

What is the difference between iptables and nftables?

iptables uses the legacy ipt_register_table() interface. nftables uses the modern nf_register_net_hook() interface. Both use the same Netfilter hooks underneath.

Why is my iptables rule slow?

Each rule adds a hook function to the array. With 1000 rules, every packet traverses 1000 hook functions. Use nftables with sets/maps for O(1) lookups.

What is conntrack?

Connection tracking — the kernel tracks all active connections (source/dest IP, port, protocol, state). Used by NAT and stateful firewall rules.

How does NAT work?

SNAT (source NAT) modifies source address at POST_ROUTING. DNAT (destination NAT) modifies destination address at PRE_ROUTING. Both use conntrack to track translations.

What is the difference between FILTER and NAT tables?

FILTER table (LOCAL_IN, FORWARD, LOCAL_OUT) decides whether to accept/drop packets. NAT table (PRE_ROUTING, POST_ROUTING) modifies packet addresses.

Conclusion

Netfilter is not a chain of linked lists — it's a compact array of hook functions. This design provides better cache locality, lower memory overhead, and faster traversal. The five hook points (PRE_ROUTING, LOCAL_IN, FORWARD, LOCAL_OUT, POST_ROUTING) provide comprehensive packet interception, and per-netns arrays enable container isolation.

For production systems, the practical takeaways are: arrays beat linked lists for hook storage, conntrack enables stateful filtering and NAT, and understanding the hook points helps debug firewall issues.

Sources

• Linux kernel source, net/netfilter/core.c, nf_hook_slow()
• Linux kernel source, include/linux/netfilter.h, nf_hook_ops
• Linux kernel source, net/netfilter/nf_conntrack_core.c, nf_conntrack_in()
• Linux kernel source, include/net/netfilter/nf_conntrack.h