"NAT Hide failure - there are currently no available ports for hide operation"

Today we finally hit the wall and run out of available free ports for NAT. In my case this was an expected error to occur sooner or later due to too many hosts hiding behind the same IP.

We solved this incident by changing the hide_max_high_port from 60.000 to 50.000 in the DB using GuiDBtool and then installed the policy. This gave us additional 10.000 ports for hide NAT.

Some useful hints to troubleshoot:
fw tab –t connections –f –u | awk ‘{print $9”,”$11”,”$13”,”$15”,”$43}’ > /tmp/connections.txt

Summary - top 20 sources
awk -F"," '{print $1}' /tmp/connections.txt | sort -n | uniq -c | sort -rn | head -20

Summary - top 20 destinations
awk -F"," '{print $3}' /tmp/connections.txt | sort -n | uniq -c | sort -rn | head -20

From R77.30 You can check the allocation
[Expert@HostName]# fw ctl get int fwx_nat_dynamic_port_allocation


Solution ID: sk103656
Solution Link: sk103656 

Other good resources for this:
Connections with Hide NAT are dropped during policy installation due to NAT port allocation failure when CoreXL is enabled
sk86401

NAT table reaches its maximum capacity on ClusterXL, which causes traffic issues
sk36708