You tried something, the problem went quiet, and you moved on. But did the fix do that, or did the problem fade on its own? To know if a solution really worked, you need three things written down: what success would look like, how bad things were before, and what happened after enough time had passed. Without them, a lucky week can pass for a cure, and a good fix can be dropped because it looked slow.
The habit is simple once it has a place to live. Problem Graph is one such place: a journal where a problem goes down in one line and each solution tried on it gets an outcome of worked, partly or failed. The method holds on paper too. What matters is that the judgment is made against a record, not against memory.
Why it is hard to know if a solution really worked
Most fixes are judged by feel. You remember the worst moments of a problem, you remember the effort you put into the fix, and when things seem calmer you credit the effort. That reasoning skips two questions. Was it really calmer, or does it only seem so? And if it was calmer, was the fix the reason?
Two well known traps sit behind those questions. The first is about numbers that swing. The second is about order and cause. Both lead people to keep fixes that did nothing and to drop fixes that helped.
Bad weeks tend to be followed by better ones
People usually act on a problem when it is at its worst. That timing matters. Statisticians call the pattern regression toward the mean: when one measure of something that varies is extreme, the next measure is likely to be closer to its usual level. Pick a bad week, and the following week will often be better whatever you do.
So a fix started during a crisis gets credit it may not have earned. The Wikipedia entry on the idea makes the practical point plainly: when you select the most extreme cases, follow-up checks help you avoid jumping to false conclusions about them. In daily life, that means a fix needs to beat the normal level of the problem, not just its worst moment.
After the fix is not the same as because of the fix
The second trap has a Latin name. Post hoc ergo propter hoc means "after this, therefore because of this": the mistake of deciding that one event caused another only because it came first. Order alone does not show cause. The error is tempting because things that happen together sometimes do share a cause, and the mind jumps from one to the other.
When a problem eases after a fix, look for anything else that changed in the same stretch. A new season, a quieter month at work, a change of routine, someone else trying their own fix at the same time. If one of those could explain the change, the fix has not yet proven itself. It may still have helped, but the record should say "partly" or "unclear" rather than "worked".
Four checks before you call it a success
The two traps share a cure: decide in advance what you will look at, and look at it in a fair way afterwards. Four checks do most of the work.
- Write the success line first. Before you start, write one sentence that says what "worked" will look like and by when. "Fewer late mornings" is vague. "Late on no more than one workday a week, for three weeks in a row" can be checked.
- Write down the baseline. Count how often the problem happens, or how bad it is, over a normal stretch before the fix. A count taken only on the worst days will flatter any fix.
- Watch for long enough. Count again over the same length of time after the fix. One good day says little. A stretch as long as the baseline lets luck even out.
- Mark the outcome, with the reason. Worked, partly or failed, plus one line on why you think so and what else changed in that time.
The fourth check is the one people skip. A short reason turns a guess into something you can come back to. Months later, "worked: late mornings went from three a week to one, nothing else changed" is worth far more than a vague memory that things got better.
A worked example: the missed morning walks
Take an invented case you can follow step by step. Someone plans a short walk before work on five mornings a week and keeps missing it.
- Problem line: "I keep skipping my morning walk."
- Success line: "Walk on at least four of five planned mornings, three weeks running."
- Baseline: over the last three weeks, 2, 1 and 2 walks out of 5. That is 5 walks out of 15 planned.
- Fix tried: put shoes and a jacket by the door the night before.
- After three weeks: 4, 3 and 4 walks out of 5. That is 11 out of 15.
The count more than doubled, so it is tempting to write "worked". Now apply the two traps. Was the baseline taken during an unusual stretch? Here the three baseline weeks were ordinary, so regression toward the mean explains little. Did anything else change? Yes: the second week after the fix was a holiday week with no commute, and it was the week with only 3 walks. The weather also turned milder.
An honest outcome: "Partly. 11 of 15 against 5 of 15 before, but the success line asked for four of five every week and week two fell short. Milder weather may have helped." That entry is useful. It keeps the fix, shows where it fell short, and points at the next thing to try, such as a plan for weeks without a commute.
Keeping outcomes where you can find them
An outcome only helps if you can find it again. Write each solution next to the problem it was meant to solve, with its outcome and its reason, and keep the failed ones too. A failed fix with a clear reason saves you from trying it a second time, and it tells anyone with a similar problem what not to repeat.
Problem Graph keeps a solution that failed on the map as a failed solution rather than deleting it. Its home page example shows the same discipline at work: a main node "Late invoices" carries three solutions, and automatic reminders are marked as settling invoices 9 days sooner. A result stated as a number against the problem is the kind of record that can be checked later.
Frequently asked questions
How long should you wait before judging a fix?
At least as long as the stretch you used for the baseline. If the problem shows up weekly, give the fix several weeks; if it shows up monthly, give it months.
What if the problem got better but something else changed too?
Mark the outcome as partly or unclear and write down the other change. If you can, keep the fix and watch a further stretch in which the other change is gone.
Is "partly" a useful outcome?
Yes. It says the fix helped but did not meet the success line, which is often the most honest result. It also points at what is left to solve.
Should failed solutions be deleted from the record?
No. A failed solution with its reason stops you from trying it again and shows what to change next time.
Judging a solution by its record
Whether a solution worked is a question about evidence, and evidence has to be collected before the answer is wanted. A success line written in advance, a fair baseline, a long enough wait and an outcome with a reason are small steps, and together they protect against the two most common errors: crediting a fix for a bad stretch ending on its own, and crediting it for something else that changed at the same time. Over months, those small records add up to a clear account of what works for your problems and what does not, and that account is the real reward of the habit.