I began this project in July 2020, it’s been 17 months of hard work and a lot of redesigns but now the moment has finally come 🙂
Achieving 100% compatibility with CRS was a core objective of this project, now that it’s done, it’s the beginning of a new phase, optimization and new features!
I want you to celebrate with me so I will provide all the tools so you can test yourself the 100% compatibility but first a few things you must understand.
- OWASP CRS regression tests are YAML files compatible with go-ftw, you can find the yaml files here.
- Coraza is not compatible with go-ftw because it requires a web server implementation and logs compatible with microseconds.
- YAML test files can be processed using the Coraza Testing package under /testing.
- There is a special repository designed to help debug go-ftw tests, it’s coraza-testsuite.
- There are 33 tests that are no compatible with Coraza, because the invalid urlencoded payloads are going to be stopped by URLENCODED_ERROR, and a standard implementation would stop that request. Coraza’s url processing is fully RFC compliant.
- There are 5 tests that are not compatible with Coraza today but are pending update on the CRS side, those rules uses quadruple backslash (\\\\) and Coraza parses them as 4 literal backslashes, the rule modifications should change from \\\\ to \x5c. (920460 and 941330)
- There is one test that doesn’t work because it uses an invalid multipart, non compliant with the RFC, which will be stopped by Coraza before triggering this rule. (932180)
- Finally, there are 3 tests that won’t work because of a bug in the rule’s regular expression (920450).
- CRS compatibility requires the coraza-libinjection and coraza-pcre plugins, also libinjection and libpcre-dev installed.
go install github.com/fzipi/go-ftw@latest git clone https://github.com/coreruleset/coreruleset wget https://raw.githubusercontent.com/jptosso/coraza-waf/v2/master/coraza.conf-recommended -O coraza.conf git clone https://github.com/jptosso/coraza-caddy sed -i 's/\/\/ _ "github.com/_ "github.com/g' coraza-caddy/caddy/main.go go install coraza-caddy/caddy/main.go wget https://gist.githubusercontent.com/jptosso/bea81ca0de225b3e09846f627abc5b74/raw/cbec9526280400bbbdb1de51460d6dc748c116ab/.ftw.yaml wget https://gist.githubusercontent.com/jptosso/bea81ca0de225b3e09846f627abc5b74/raw/cbec9526280400bbbdb1de51460d6dc748c116ab/Caddyfile caddy start -adapter caddyfile -config ./Caddyfile go-ftw run -d coreruleset/tests/regression
To test the rules you must type the following commands:
git clone https://github.com/coreruleset/coreruleset go install github.com/jptosso/coraza-testsuite@628b960 wget https://raw.githubusercontent.com/jptosso/coraza-waf/v2/master/coraza.conf-recommended -O coraza.conf coraza-testsuite run -crs -d ./coreruleset/tests/regression -r "./coraza.conf,./coreruleset/crs-setup.conf.example,./coreruleset/rules/*.conf"
And the expected result should be:
203 profiles were loaded Skipping 920120-4 Skipping 920120-6 Skipping 920120-7 Skipping 920181-1 Skipping 920240-1 Skipping 920240-5 Skipping 920240-6 Skipping 920460-1 Skipping 920460-2 Skipping 920460-3 Skipping 920460-4 Skipping 921150-1 Skipping 921160-1 Skipping 932140-3 Skipping 932180-2 Skipping 941110-6 Skipping 941130-2 Skipping 941130-4 Skipping 941130-6 Skipping 941130-9 Skipping 941130-10 Skipping 941130-11 Skipping 941130-12 Skipping 941130-14 Skipping 941130-16 Skipping 941280-2 Skipping 941330-1 Skipping 942100-10 Skipping 942100-13 Skipping 942150-6 Skipping 942260-6 Skipping 942260-17 Skipping 942490-17 Failed:  Passed 2562/2562 (100.00% passed)
You can debug a single test of a pattern of tests using the -i option to pick the tests and the –debug option for debugging, for example:
coraza-testsuite run -crs -d ./coreruleset/tests/regression -r "./coraza.conf,./coreruleset/crs-setup.conf.example,./coreruleset/rules/*.conf -i "942490-" --debug
Thanks to everyone who has helped to make this possible 🙂 And feel free to post any issue or comment about the tests.
Special thanks to @fzipi for his help debugging (and being a super rubber duck debugger 😂)