Mention fnox

[domenkozar]

Recently mise rebranded secretspec into fnox, implementing some of the things talked in the issues here.

Leaving this around so we can have a comparison page in the docs.

[polarathene]

Can you elaborate please?

For example there is no clear shared source such as a fork:

secretspec/secretspec/src/provider/onepassword.rs

Lines 238 to 243 in da72913

	pub struct OnePasswordProvider {
	/// Configuration for the provider including auth settings and default vault.
	config: OnePasswordConfig,
	/// The OnePassword CLI command to use (either "op" or a custom path).
	op_command: String,
	}

• Related initial commit of fnox.
• Related engagement with mise fnox announcement.

---

Related context

Recently the ubi backend of mise became deprecated in favor of their own github backend, which seems like it'll be eventually dropped despite this earlier July comment.

I am not too familiar with mise yet, but assume by "rebranded" it may have been similar to the ubi scenario? Was mise using secretspec prior and then adopted their own LLM assisted alternative that may have then inferred similar functionality/API? The dev did describe their process:

I basically just pointed an llm to the existing secrets functionality in mise and it was able to make it in a new place.
It took me longer to setup that secret stuff than it did to write fnox.

secretspec itself is also using LLMs for development apparently? (again I haven't looked into that much) You've both got Claude setup, so perhaps it's not too surprising if there's overlap, but as noted above it's not exactly a fork/copy of the project (although could be considered more concerning, less trustworthy).

Less concerning (collapsed for brevity)

While looking into mise I also came across this mise-plugins/vfox-semver package which by repo name implies some association to vfox tooling (EDIT: Conforms to vfox plugins which rely on Lua), as does the initial commit message:

Converted asdf-semver plugin to mise vfox-style plugin format.
Includes support for listing, downloading, and installing semver tool.

Reference: asdf-semver plugin (a wrapper to this semver project).

Despite the URL for the project resolving to mise-plugins/vfox-semver repo, the README since it's init commit has referred to an alternate name (which resolves to the vfox name):

Another maintenance issue from the dev is related to discouraging vfox plugins, while encouraging ubi adoption, which is deprecated. The developer seems highly reliant upon LLM tooling:

[theutz]

I was involved with fnox as an early adopter, because the pain around secrets has been one of my biggest productivity sucks. I was a heavy mise user, and fnox seemed like a huge leap in the right direction.

Right away, though, I started noticing very odd behavior in the program. Inconsistencies in the CLI flags, some commands deleting data on accident, etc. I kept sending detailed feedback in the discussions, and they were largely addressed right away.

But often, in addressing them (w/ significant help from Claude), new problems would pop up. Or old ones would resurface. Or new varieties of flag combinations would appear.

I'm not a Rust programmer, but I'm a heavy terminal user. And watching an expert dev like that making such fundamental mistakes with LGTMs to an LLM is one of the things that has caused me to be much more skeptical of AI-assisted coding than I was before.

[polarathene]

I'm too skeptical of projects leveraging AI heavily these days. It's probably going to become harder to avoid but there's often just too many problems with adopting them that it's exhausting or taking a risk.

---

And watching an expert dev like that making such fundamental mistakes with LGTMs to an LLM is one of the things that has caused me to be much more skeptical of AI-assisted coding than I was before.

Yeah and that's with someone that apparently has a rather successful background/reputation as a professional dev.

I assume it's due to the tradeoff for velocity given how much they keep pushing development along.

Another experienced dev embracing AI landed over 40k commits in less than a year and has recently had some rather popular projects deployed with users (less technical kind, oriented at automation via AI) that were exposed to have vulnerabilities like:

• XSS (Users could upload an SVG image with embedded JS that wasn't sanitized and then served on user content pages from the same origin as auth session cookies, ran JS to steal JWT refresh tokens). This would display on various pages listing results with the SVG as a thumbnail for the associated content page, which then gains access to the logged in users account and provisions an API token for continued use, if the user had any content uploaded/shared already that'd get the SVG malware to widen the infection rate and the actual content (automating AI agents IIRC) could likewise be modified to exploit all users of the community content.
• Publicly exposed Supabase DB IIRC with no RLS (row level security) enabled. API keys of all registered users available to grab. This was a separate service but gave access to additional systems/accounts through those API keys.
• Full write access to Github account IIRC from a desktop app/widget that only needed scoped read access. That was reported weeks ago and the dev continued to have activity elsewhere in the project (and presumably distracted by addressing the publicly disclosed vulnerabilities of the more popular web products), so they're possibly not even aware of the reported issue just remaining open on GH with no engagement.

I suspect that review becomes a friction point with the amount of churn and that they place a bunch of trust into their tooling being reliable most of the time that become more laxed/trusting during review? Signal noise is probably high too.

The inexperienced vibe coders that don't have any dev expertise are more worrying and prone to the risks, especially with software that can look appealing on the surface and be difficult to distinguish that AI was heavily relied upon. It's at least a bit easier in the OSS space to discern this, with apps/libraries that can be really attractive but have glaring issues when it comes to relying upon them or the fact the rate of abandonment is high.

---

Just yesterday I was looking for a library to use for some functionality on crates.io, and one I looked at was a fork of an existing library that turned out to be a vibe coded modification to "replace OpenSSL with native Rust deps"

• There was no engagement with upstream about collaborating/upstreaming the change and on further inspection the transitive OpenSSL dependency was only relevant for running an example or in a build.rs script to support cloning a repo to acquire a single file from a fixed commit (hard-coded git tag from Dec 2024).
• The vibe coder did an elaborate workaround with broad changes when all they would have needed was the openssl dev package installed in their build environment, or to replace the clone in build.rs with a vendored copy of the headers file that was needed.

It's quite concerning what's going on and we'll need to be much more cautious. Even some traditional projects are starting to embrace AI and make poor decisions through it 😓

This isn't to say AI assisted development is bad in itself, it can be done well and be great but I'd assume a good bulk of the usage will be prone to the concerns raised.

Vibe coding notably enables a very different demographic of builders into the OSS community that have different priorities and ethics.

---

I was a heavy mise user, and fnox seemed like a huge leap in the right direction.

Right away, though, I started noticing very odd behavior in the program. Inconsistencies in the CLI flags, some commands deleting data on accident, etc. I kept sending detailed feedback in the discussions, and they were largely addressed right away.

But often, in addressing them (w/ significant help from Claude), new problems would pop up. Or old ones would resurface. Or new varieties of flag combinations would appear.

Yeah that's an observation I've had too. Mise notably uses CalVer for it's releases which means you can't actually depend upon it to not have breaking changes (anything treating it's releases as semver will also be problematic), but even then I don't know how reliable semver is to be respected by products that are primarily AI driven in development.

I don't have any opinion on this secretspec project, but fnox is just one of many from the mise dev. AI aside, it's difficult to trust projects that are security focused from someone who's not that specialized in the domain (delegating expertise to LLM advice rather than being clued up?) but as I've not looked into the project itself that much I can't really criticize it properly.

What you describe as issues you experienced with using it though is concerning given it's description implies a goal of the project is to avoid those kind of issues in the first place. Devs that use AI as an abstraction to the codebase and don't carefully monitor those kind of changes (or care to respect semver) should be a red flag (as great as the project may be compared to alternatives, this is a pretty big deterrent).

[theutz]

but even then I don't know how reliable semver is to be respected by products that are primarily AI driven in development.

You know, that's an excellent point I hadn't thought about before. If project maintainers are less-and-less engaged with the details of what their software is doing, they are more-and-more likely to introduce breaking changes without realizing it. And even if those changes don't represent real "bugs" or vulnerabilities, obviously a proliferation of breaking changes without warning is a massive net-negative for collaborative development.

If this kind of practice becomes widely adopted, the contracts that most package managers are built on would become completely meaningless. Figuring out which version of which dependency you need to make your product would be an absolute nightmare. At which point, you might be tempted to just have the LLM "write-up" the functionality for you from scratch (read: steal it from previous training data unattributed).

Which is, of course, ideal for the business models of the LLM companies, as it creates demand for a higher volume of tokens and increases dependence on their product.

But, my god, what an awful future awaits us if we don't resist this madness today?!?!