Hello all, I created QUALITY.md to help build a holistic quality evaluation process for my projects. Turns out it's also ideal for loop engineering. I'm hoping this provides a valuable contribution to the conversation around quality and craft and having AI help us in the effort. I hope to shift the mindset from a reactive/review/repair mindset to a proactive care mindset.
Give it a go. I look forward to your thoughts/comments/feedback!
Website: https://getquality.md GitHub: https://github.com/qualitymd/quality.md
Is this really where we've landed? I refuse to believe that any of this markdown insanity will continue indefinitely.
It's insane to me that the "fix" for AI errors is adding more "PLEASE PLEASE DO BETTER" to the prompt
If it makes you feel any better, the Markdown part is optional (and has no semantics). Somehow it feels about right that the Markdown file can actually just be a YAML file with the wrong extension.
(Actually, to be more specific, a YAML file with no directives, explicitly-signalled start-of-document-content, and followed by a second null document. I will note that frontmatter syntax is not specified; the non-normative Appendix B is the only place that suggests it means prefix and suffix --- lines. And no, frontmatter is not part of Markdown, or CommonMark, and is in fact incompatible with both. And it’s invalid YAML too, the end-of-frontmatter line should be ... to indicate end of document without starting a new document.)
QUALITY.md feels similar to CONSTITUTION.md
Looks like unless something better comes up, we'll be stuck with it for a while.
I find markdown useful for repo-specific conventions, especially skills.
it's looking like llms are interpreters, and markdown plus english text is the language of choice to run non deterministic programs on it
I thought the same about Yaml and Kubernetes/Helm…
It already refuses to read the AGENT.md/CLAUDE.md files, what's the point of giving it even more markdown it won't read until you yell at it.
it is until we define real consistent deterministic gates and protocols. It really is a symptom of the lack of concerted effort. Everyone has a personal preference on how to shove the context and most of them are just "here's some good text I've found to work in my context"
The one thing I do not understand is that here you say:
"Ensure stakeholders are aligned on what matters most and why"
But it is instructions for LLMs, right? A way to describe something that the humans know and the LLMs don't.
LLMs literally cannot be stakeholders, by definition.
THe problem is that humans often don't know - this is as much about encouraging getting the humans aligned as the agents. Completely agree agents really aren't stakeholders. Fine point. I'll update description to clarify ... thank you!
Not OP, but it seems to me the idea is that stakeholders can collaborate and come to consensus on the contents of QUALITY.md.
Whats the revenue model for this NBPaaS? (No Bugs Please As A Service)
I'm less interested in this than in what people are willing to aggressively trade off against in order to get the stuff they truly care about.
For example, readability. Where are the developers out there saying "I am very willing to sacrifice a lot of readability to get even a small improvement on e.g. abstraction cleanliness", and sticking with it?
Or "performance can take a huge hit at the cost of being dead easy to read and reason about". Coming up with a list of abstractly good-sounding qualities is just prosocial signaling without knowing what you're willing to sacrifice. There should be a FUCKIT.md that enumerates these.
OP here. You're spot on. Trade-offs matter. The trade-offs are implied by the selection of what quality factors/attributes are selected and their requirements. A statement like "performance can take a huge hit at the cost of being dead easy to read and reason about" can sit right there in the QUALITY.md as a comment or in the markdown body.
This is perfectly encapsulated in xkcd's "Standards" strip [https://xkcd.com/927/].
Useful Nice
Pure slop.
What?