Falco + Nginx Plugin Development: Falcoya Days 51-56
〜 Test Report Publication, UI Fixes and Internationalization, and Attack Verification Re-Challenge 〜
Review of Previous Days
Days 45-50 were filled with E2E test enhancement, HTML report fixes, and preparation for attack log verification. Rather than flashy achievements, the focus was on mundane improvements like Nginx log formatting and Falco rule adjustments. It was a period that further strengthened the culture of "recording failures and turning them into assets."
Then came day 51 and beyond. I proceeded with E2E test report publication, UI fixes, i18n support, and attack verification re-challenge.
Day 51 (08/30) — End-to-End Test Report Publication
On this day, we finally published the End-to-End Test Report (Phase 1). 👉 Test Report
We fed 14 scenarios into the system and published the results as they were: 12 successful detections (SQLi: 5, XSS: 7) / 2 undetected. For each scenario, we listed "triggered rules, Falco output, success/failure determination" so that outsiders could also verify what was caught and what was missed.
"Publishing not just successes but also undetected cases." It's scary, but it's the honesty of OSS. Falco plugins are not omnipotent, but I realized that the stance of improving with transparency is our true strength.
Learning
OSS honesty means publishing not only successes but also undetected cases. The stance of continuous improvement with transparency is our true strength.
Day 52 (08/31) — A Step Toward Internationalization
The next challenge was i18n (internationalization support). Work began to display reports in both Japanese and English.
As work progressed, we found missing translation keys and organization oversights, causing fluctuations in screen display. We fixed the shortages one by one while organizing the UI.
What I learned was that internationalization is not just translation work but design itself. To provide a consistent experience even when switching languages, it needs to be built into the system from the beginning.
Learning
Internationalization is not just translation work but design itself. To provide a consistent experience when switching languages, it needs to be built into the system from the start.
Day 53 (09/02) — Attack Verification Re-Challenge
We again fed SQLi and XSS logs and tested Falco's response. But the results didn't go as expected.
While some detections worked, false positives and false negatives occurred, leaving accuracy issues. I recorded the results in PROBLEM_PATTERNS.md and organized them as improvement points.
"Failures that can't detect" and "failures that detect too much" — both are fatal for users. How to balance these two became the next major challenge.
Learning
The balance between false positives and false negatives is crucial. Both are fatal for users, and adjusting these two becomes the next major challenge.
Day 54-55 (09/03-09/04) — False Positive Adjustment
These two days were spent on how to prevent false positives. If we loosen the regular expressions too much, we'll miss attacks; if too strict, false positives increase. We repeatedly adjusted conditions and added the content to integration-test-requirements.md.
We organized specific improvement points for the next verification.
Learning
Detection accuracy adjustment is a delicate balance. Understanding the trade-off between regex strictness and detection range is important.
Day 56 (09/04) — Multi-layered "Net" and Integrated Documentation
On this day, based on all previous adjustments, we massively expanded rules and attack patterns and documented the entire scope.
The deliverable was IMPLEMENTED_RULES_OVERVIEW.md. This includes:
- Complete list of 37 implemented rules
- Detailed catalog of 810+ attack patterns
- Implementation status by category
- SQLi: 290
- XSS: 160
- CMD Injection: 150
- Path Traversal: 100
- Emerging Threats: 60
- Detection capabilities and performance indicators for each rule
From Phase 1 (6 rules, 18 patterns) to 37 rules and 810+ patterns in just a few weeks. Layer upon layer of nginx log rules, a meticulously designed "net" to catch diverse malicious behaviors. This is the essence of using Falco and the greatest achievement we've built as OSS.
Learning
Falco's essence is the multi-layered "net". With meticulous design of 37 rules and 810+ patterns, diverse attacks can be reliably captured.
Tasks Completed in Days 51-56
- Publication of End-to-End Test Report (Phase 1)
- Verification of 18 attack patterns with 6 Falco rules
- Explicit publication of both successes and undetected cases
- i18n support (fixing missing translation keys and organization)
- Attack traffic verification re-challenge (confirming false positives and negatives)
- False positive adjustment and recording in integration-test-requirements.md
- Massive expansion of rules and attack patterns (37 rules, 810+ patterns)
- Creation of integrated documentation IMPLEMENTED_RULES_OVERVIEW.md
Created/Updated Documents
integration-test-requirements.md
→ Added false positive countermeasure condition adjustments
PROBLEM_PATTERNS.md
→ Newly added "false positive issues" and "undetected scenarios"
IMPLEMENTED_RULES_OVERVIEW.md
→ Recorded the complete scope of 37 rules and 810+ attack patterns
Summary
Days 51-56 had "How to demonstrate OSS transparency" as a major theme. Particularly with the E2E test report publication, we demonstrated OSS honesty by publishing results of 6 rules and 18 patterns including both successes and undetected cases.
Then on 9/4, we expanded implemented rules to 37 rules and 810+ patterns and documented the entire scope in integrated documentation. The essence of Falco plugins is the ability to detect diverse attacks by layering meticulous rules based on nginx logs.