Six months ago in Jan I ported JustHTML to PHP. I had also released a CLI for it and published it via Homebrew to make document parsing, selecting DOM nodes and dumping to markdown super easy. Most of the heavy lifting at the time was done by ChatGPT 5.2 Codex, a fantastic model that did not complain about rate limits even once with my $20/month plan.
With Fable 5 and ChatGPT 5.6 Sol being released, July was a good time to revisit this library, see if these new models could find bugs, improve performance or improve the ergonomics of the package.
Model and Harness
I worked mostly with ChatGPT 5.6 Sol in the Codex app (now simply called the ChatGPT app). Initially I set the reasoning level to Extra High, which was the highest available option in the app. But when I did that, it became like Fable 5. i.e., I would get rate limited after only a couple of conversation turns. So I switched to Sol Medium as the primary workhorse.
I used Fable 5 via the Claude GUI app for reviewing Sol’s work. More on this below.
The other big change from January was YOLO mode. Six months ago, I would review each permission request. This time around, I basically gave the models carte blanche. All the work happened with “Bypass permissions” (Claude) or “Full access” mode (ChatGPT). I have now crossed the Rubicon where I am not reviewing the AI’s actions closely any more.
Development workflow
The development process I am settling on is to use both Fable and Sol and have them critique each other’s work to arrive at a plan that combines the best of both. It goes somewhat like this:
- Ask both Fable and Sol separately to do a task e.g., find bugs, opportunities for performance improvements, ways to improve documentation etc.
- Skim through their responses. Ask them to write their findings/plan to a .md file.
- Ask Fable to review Sol’s .md file and see if it contains findings Fable missed. Then ask Fable to update its .md file.
- Ask Sol to review Fable’s updated .md and save its critique in a separate .md file.
- Ask Fable to review the feedback on its plan by reading the .md file that Sol wrote, and then update its plan.
- Loop steps 4 and 5 a few times until I’m satisfied there are no major issues.
- Optionally ask Grok 4.5 to review the final plan.
- Get Sol to implement the plan. Fable would just kick me out with rate limits. Sol is a great workhorse. It gets shit done.
The initial outputs from Fable and Sol are only about 60% as good as the final version after the review/update loops. Based on the quality of output I have seen and the results I have gotten from having these models review each other’s work, I would not rely on any single model ever for anything important like planning or architecture design. No matter how good the model is.
In fact, I suspect we can get similar levels of performance using a panel of much cheaper, less capable models. Fireworks AI posted yesterday that by routing intelligently between Kimi K3 and Fable 5, they were able to surpass both models in performance. The review/update loop-based workflow I am describing is similar but different from model routing:
- We don’t choose which model to route a specific task to.
- Other than the initial kickoff prompt, we don’t send the same prompt to different models. The approach is NOT to send the same prompt to different LLMs and then use another LLM as a judge to choose the best output.
Markdown files are how I established communication between the two models. But I had to switch back and forth between two apps to communicate with the two LLMs. A new harness for this multi-model development process could eliminate app switching and provide a smoother development experience. Perhaps something to build next. We are lucky to be living in such interesting times.
Code review
I have stopped reading code on vibe-coded software where I never wrote any code in the first place. For pre-2024 software where I am familiar with the code base because I wrote it, I still tend to review the git diff in Sourcetree (the only Atlassian product that is enjoyable to use) before committing the changes. But it’s mostly for me to update my internal weights than to check if the AI did a good job. These models are good at catching each other’s mistakes so I don’t have to do that.
JustHTML-PHP July updates
The html5lib test suite is pretty thorough so there weren’t bugs in the core document parsing and tree building. But both models found bugs in the other parts of the library that implemented the interfaces where developers would actually do something useful with the DOM.
JustHTML is significantly slower than PHP’s built-in DOM\HTMLDocument because JustHTML is written in PHP and cannot compete with something written in C. However, JustHTML has three advantages:
- full HTML5 compliance so it handles malformed HTML like a browser would
- JustHTML works for PHP 7.4+ while HTMLDocument is PHP 8.4+
- easy-to-use interfaces for selectors and to convert the information to text/markdown.
I did not want to give up on performance completely though. So JustHTML PHP’s July release makes a lot of performance optimizations. Notably, for use cases where you don’t necessarily need to traverse the entire DOM but only need some elements with early exits, the new Stream::select() interface offers a 2x speed improvement.