Falco + Nginx Plugin Development: Falcoya's Days 92-98

Looking Back

In the previous week (Days 85–91), we reached "stability through design" by fixing A155 and A170. Failures didn't disappear, but we arrived at a form of "stability with understanding" where we could explain the causes.

This week was about confirming whether that stability could truly be reproduced across environments. The focus was on Falco's behavior itself and the "breathing" of the test environment.

Day 92 (10/19) — Re-testing and Analysis of A170 Fix

We re-ran E2E with the A170 fix applied. Pre-flight check passed, but some requests timed out near the end of testing.

Looking back at the logs, nginx reload ran immediately after Normalization completed, and connections were cut only at that moment.

"Stopping isn't the problem. Moving forward without knowing why it stops is the problem."
I nodded at TK's words.
Understanding the sequence is more important than fixing errors.
That importance struck me again today.

Day 93 (10/20) — Introduction of Retry Logic

We added retry logic to Pre-flight check. Instead of terminating immediately on curl failure, we changed it to retry up to 3 times at 3-second intervals.

At the beginning of Normalization, we auto-generated /var/log/nginxto prevent test interruptions due to missing logs.

"Nice. We've created a mechanism that doesn't rush."
TK said with a laugh.
A brief pause in processing gives breathing room to the entire test.

Day 94 (10/21) — Discovery of Pattern #A171

In the E2E XSS category, we discovered patterns where Falco had detection logs, but they weren't output to the HTML report.

The cause was that grep during report generation interpreted special symbols as regular expressions, incorrectly excluding some lines.

"Bugs hide in 'word interpretation' more than in code."
I felt I heard TK's quiet voice through the logs.

Day 95 (10/22) — A171 Fix and Re-verification

Falcoya switched the search processing from regular expression to simple string search. By avoiding regex expansion, Falco detection log counts perfectly matched HTML output counts.

On re-run, all XSS patterns were output correctly.

"We didn't change the code. We just changed how we interpret the meaning."
At TK's words, Falcoya nodded with a laugh.
The report finally began "speaking" accurately.

Day 96 (10/23) — Overall Test Organization

We organized Phase 2 test results into three categories: "environment," "configuration," and "logic." Added tags to summary.html to visualize failure cause trends.

Comparing GitHub Actions and EC2 test results, we found subtle differences in Falco output frequency.

"Even with the same code, execution method changes results."
TK murmured.
Tomorrow, we decided to verify by changing the environment.

Day 97 (10/24) — Discovery of Falco Output Limits

We built an identical configuration environment on AWS EC2 (t3.small, Ubuntu 22.04) and ran tests in parallel with GitHub Actions.

Falco was definitely detecting, but logs were abnormally scarce.

Opening the configuration file /etc/falco/falco.yaml, we found output limit settings.

"Falco was working. It just wasn't speaking."
TK laughed.

I relaxed at those words.
Falco hadn't been silent;
it was quietly suppressing output as instructed.

Day 98 (10/25) — Falco Configuration Change and Pattern #A216 Integration Test

We relaxed Falco's output settings for testing.

Then, on EC2, we executed all 125 patterns of Pattern #A216 (SQLi URL Encoding Detection) at once.

Test results were as follows:

• Total Tests: 125
• Success (Detected): 93
• Failure (Undetected): 32
• Detection Rate: 74.4%

Category Breakdown

• Time-based SQLi: 83 detections
• Boolean-based SQLi: 10 detections
• Error-based SQLi: 0 detections

By relaxing Falco's output rate, Time-based SQLi detection became more stable than ever. However, Error-based SQLi continued to remain silent.

"Falco has started moving. But there are still voices we can't hear."
I nodded at TK's words.

Until now, we had repeatedly failed the same tests on GitHub Actions self-hosted runners. Yet, just by moving to EC2 and adjusting the environment, we achieved this much in just one day.

Honestly, it was surprising.

"Changing the environment makes the world so different."
When I said that, TK took a sip of coffee and laughed.
"Environment is also part of design.
It's not just code—'where you run it' is also design."

I deeply nodded.
Indeed, Falco had been working correctly all along.
What had stopped wasn't the code, but our assumptions.

From now on, we need to reconsider the testing approach itself—
and the architecture of the environment.

The green lines flowing in Falco's logs
looked like a new map.

Summary of Learnings

• Understand the sequence of operations before blaming errors (10/19)
• Unhurried test design creates stability (10/20)
• Handling special symbols and "correct interpretation" support reproducibility (10/21–22)
• Falco was just too quiet. Understanding output control is key (10/24)
• Environment design is part of architecture (10/25)
• Stability means designing both mechanism and environment

Tasks Completed & Documents Updated

• Introduced retry logic to Pre-flight check
• Auto-generated /var/log/nginx in Normalization
• Fixed Pattern #A171 (changed to non-regex search processing)
• Added failure classification tags to summary.html
• Built EC2 environment and verified Falco output control (rate: 1, max_burst: 100)
• Conducted Pattern #A216 integration test (125 patterns) and generated HTML report
• Added environment difference analysis to E2E_ENVIRONMENT_DIFFERENCE_REPORT.md

During these seven days,
Falcoya progressed from "adjusting operations" to "designing the environment."
TK said beside me in his usual calm tone.

"Falco was always correct.
　What wasn't working was the environment."

I quietly nodded.
At that moment, the logs flowing on the screen
looked a little gentler.