We recently learned that public-facing memcached instances have been used in amplification attacks. Here is a relevant quotation from Arbor Networks [1]:

We have observed a considerable uptick in memcached reflection/amplification attacks ranging in size from a few hundred mb/sec up to 500gb/sec and larger. The amplified attack traffic is sourced from UDP/11211, with a packet size of 1428 bytes (1442 bytes with layer-2 Ethernet framing included), and no fragmentation (memcached segments large responses at layer-7, as does ntp). The attacker typically ‘primes’ a given set of memcached reflectors/amplifiers with arbitrary-length key/value pairs, and then issues memcached queries for those key/value pairs, spoofing the IP addresses of targeted hosts/networks. Both the priming queries and the attack-stimulus queries can be directed from source ports of the attacker’s choice to UDP/11211 on abusable reflectors/amplifiers, meaning that the attacker has full control of which destination port is targeted on the destination hosts/networks.

It should also be noted that memcached priming queries can also be directed towards TCP/11211 on abusable memcached servers. TCP is not currently considered a high-risk memcached reflection/amplification transport as TCP queries cannot be reliably spoofed.

According to the sources I have read, legitimate memcached traffic across the public Internet is rare, and the simplest fix for this issue is to block both inbound and outbound UDP and TCP traffic on port 11211.

If you are running memcached you may wish to consider binding it to localhost, rather than the default behavior of binding it to all ports. John at Nuclearfallout Enterprises, Inc. has shared the following tip for doing so:

On CentOS/RHEL,

  1. Open /etc/sysconfig/memcached in your favorite text editor.
  2. Change the line currently reading OPTIONS=”” to OPTIONS=”-l 127.0.0.1”
  3. Save the file and exit the editor.
  4. Restart memcached with this command: /etc/init.d/memcached restart

On Ubuntu/Debian,

  1. Open /etc/memcached.conf in your favorite text editor.
  2. Locate the line containing the -l parameter and adjust it to read “-l 127.0.0.1”. If there is no line with a -l parameter, add one at the end of the file.
  3. Save the file and exit the editor.
  4. Restart memcached. To do this, you may need to use command “service memcached restart” or “systemctl restart memcached” depending on your version of Ubuntu.

Further Reading

  1. memcached Reflection/Amplification Description and DDoS Attack Mitigation Recommendations
  2. The memcached amplification attacks reaching 500 Gbps
  3. Open Memcached in Zimbra 8.6.0_GA_1153
  4. In-the-wild DDoSes use new way to achieve unthinkable sizes