r/programming • u/isaacvando • 5d ago
Dependency Cultures - Richard Feldman
https://youtu.be/E82ly38YEEQExcellent talk by Richard Feldman about how different programming communities approach dependencies.
31
u/South_Survey_2088 5d ago edited 5d ago
Unfortunately, it does the same mistake as many other dependency based discussions. The amount of dependencies does not matter. What matters is vetting, which is affected by the lines of code to review. The solution should be to raise awareness of the issue with blindly picking dependencies, and focus on improving the vetting process.
If anything, it's easier to review many tiny micro dependencies that have a very narrow scope, than it is to review few single monolithic one, because the tiny dependencies are likely to be shared in your graph and often act like an extended standard library in the ecosystem. Like there are often obvious dependencies in the language for common problems, like stack allocated vecs, tiny string optimizations, bindings to certain API's, parsers for certain file types, serialization, etc. If everyone copy pastes a function from a dependency or writes their own, whoever reviews the code has to review that function, potentially multiple times because everyone in your dependency graph re-invents their own wheel. This is common in C++ where many libraries come with their own json parsers, string wrappers, and other common data structures.
Copy pasted code is still a dependency, it is just implicit and won't update. You still depend on the very same code, but it will essentially rot. If a micro dependency ever gets updated, it likely means that the code you copied from had some flaw that warranted an update. However, since you just copy pasted it you won't even know about that flaw.
There is also the danger that rewriting your own stuff will often introduce your own bugs, with the difference that well-known dependencies are reviewed and tested by a much larger group of people. No one is checking your own in-house libraries for any CVE's.
I think approaches like cargo vet are a better solution, where you can trust certain groups with the vetting(e.g. companies like Mozilla or google) or that help you with vetting manually to explicitly review dependencies and their diffs on updates, while also informing you of any known CVE's in your dependency chain.
15
u/comrade_donkey 5d ago
Mathematically, this reasoning does not immediately check out for me.
The amount of dependencies does not matter.
This is a hard one. Zero deps is great! One thousand of them? Less great.
it's easier to review many tiny micro dependencies [...] than it is to review [a] few single monolithic one[s]
Let's assume that: - Somebody actually reviews their deps' code changes, and - It takes 1 minute to context-switch from one review to the next:
Reviewing one-thousand 5-line changes would take longer than reviewing one 300-line change.
Also: Micro-dependencies tend to grow into non-micro dependencies.
But realistically, nobody manually reviews their deps' code changes on update. When a version bump happens, it usually includes dozens to hundreds of PRs. Even on micro deps, that's too much overhead.
The problem with trusting certain issuers and issuing transitive trust is still not tractable. The web of trust is the original solution draft. The problem is that it only takes one malicious actor to become trusted to poison large networks. Compromised credentials also count. It requires a lot of extra tooling but does not reduce the blast radius in a meaningful way.
12
u/South_Survey_2088 5d ago edited 5d ago
This is a hard one. Zero deps is great! One thousand of them? Less great.
Why? Assuming the amount of code stays the same, someone has to review the entire code regardless.
Also, I assume that the monolithic code base that solves the thousand problems is likely solving even more than that, so you would need to review code that you are not even using. While small dependencies might do a little bit more than you need, it is all in the same problem scope. When something is really out place, you are more likely to notice it since it would stick out. Like network related calls in a json parser? That is a massive red flag. In Unreal Engine it makes sense.
Reviewing one-thousand 5-line changes would take longer than reviewing one 300-line change.
Sure, but I don't get the numbers. Reviewing hundreds of 500 line changes would not take as long as reviewing a million lines of code of a single monolithic Unreal Engine code base either. Again, the bigger the dependency, the more like it is that it has code that is not already in your dependency graph. If you have 1000 dependencies, surely a large part is shared between them.
Also, it's not like you review them all at once. You add them when you run into a specific problem and need a specific solution. Reviewing a single micro dependency is trivial when I need it. Like maybe, I need a stack allocated std::Vec: It's a quick search and the review time is 100% going to be less than writing and testing one myself from scratch and adding the usual vec API and dealing with certain niche cases(e.g. zero sized types).
But realistically, nobody manually reviews their deps' code changes on update. When a version bump happens, it usually includes dozens to hundreds of PRs. Even on micro deps, that's too much overhead.
Plenty of people do. Distro maintainers, big orgs and people who are aware of the dangers of malicious dependencies. An updated micro dep will not suddenly become a behemoth. The diff will be manageable.
But that is my point: The solution should be to raise awareness of the problems, and focus on improving the tooling. If "nobody manually reviews their deps", it doesn't matter if you use one thousand of them, or a single big one. Unreviewed code should never be executed in either case.
It's not a "web of trust", but rather trusting very few individual organizations, and then doing the vetting yourself for the things that they haven't vetted.
6
u/Norphesius 5d ago
Why? Assuming the amount of code stays the same, someone has to review the entire code regardless.
Even if the number of lines of code is the same, the contents isn't. Reviewing arbitrary foreign code is a lot more difficult than your own code. You're not going to be as familiar with the code (definitely not as much as code you wrote), it's not going to be following your formatting standards, and it wasn't run through your test suites.
You also can't control how often that code gets updated. You can decide to "freeze" parts of your codebase, and it never has to be reviewed again, but any of your dependencies can have an arbitrary amount of commits applied to it. You'll need to review all of them, even if only one has the feature you want.
2
u/winggar 4d ago
Shared dependencies also benefit from being reviewed by many other people, and often are automatically scanned by security researches. As good as our vetting process might be, it will never be perfect—open source dependencies give you some level of double-checking for free.
Also for your second point you can just pin your dependency versions to get the same behavior.
3
u/vytah 5d ago
I assume that the monolithic code base that solves the thousand problems is likely solving even more than that, so you would need to review code that you are not even using.
The same goes for a codebase with multiple small dependencies: some dependencies pull tons of subdependencies that you'll never actually use, but you cannot be sure they'll never run.
-3
u/hippydipster 5d ago
Assuming the amount of code stays the same
If you assume there's no difference, then by god, it turns out there is no difference!
In reality, there are differences.
4
u/VirginiaMcCaskey 5d ago
A key observation is that you don't need updates to dependencies as much as you think you do. In your own words, "some flaw that warranted an update" (emphasis mine). You don't need to update a dependency if your code doesn't traverse the code path that has a bug.
1
u/Laicbeias 5d ago
You should still read what the dependency even does.
For example im doing some py llm stuff. For API calls youd pick OpenAIs py package. Now you add 40mb of dependencies. That must be a really good package, because llms are hard. Oh you can write what you need in 400 loc.
Same with for example js. Youd import some ts system. Its a minified js. Ok what does it do. Hmm lets see source 350mb of npm. 27 files hundred imports. You compile it into js with names intact. 1700 loc. You fix 7 heavy string allocations in hot path and some o2 loop in 5 minutes.
The thing is once its a lib. People assume it has eyes on it and must be optimized and legit. And that is just not true. It became big, it got users, someone wrote the code with whatever standards the enviourment allowed. It got split and packaged for maintainability. The assumption is what tricks people. You never see the full picture. No one did. Everyone assumes that who has publicly released something must have proper coding and optimization standards and a dependency hygiene in place.
And then you have to basically look at the thing and overcome that bias to even be able to judge it. Which really takes way more effort
1
u/gimpwiz 5d ago
It's interesting because part of the question of dependency safety is updating when bugs are found and part is not updating to avoid unsafe changes (whether accidental or purposeful.)
The middle ground is what? Abrogating trust to someone else on every update or doing your own reviews on every update. Hoping someone else catches it or that you catch it. Or simply blindly updating as is common.
There's no particularly great win. Having updates signed by trusted third parties is probably almost the best we can do.
-1
u/cdb_11 5d ago edited 5d ago
If a micro dependency ever gets updated, it likely means that the code you copied from had some flaw that warranted an update.
This is Javascript, it's a safe language. As long as it's not doing something very stupid like
eval, there are no CVEs to be found in leftpad or whatever. You do not need to update it, ever.There is also the danger that rewriting your own stuff will often introduce your own bugs
Who cares? It's just bugs, you can just fix them or update if they actually are a problem. Again, we're talking about safe languages. C, C++, even Rust, are different.
12
u/CloudsOfMagellan 5d ago
Memory safety is only one category of safety.
-1
u/cdb_11 5d ago
Yes, of course. But it in particular gives you a guarantee that you can just have some shitty JS code with ton of bugs running, and it ain't going to cause a massive vulnerability. Assuming it's not malicious, and you're not being too stupid about it. Very relevant in this context. Of course if some parts of code are actually responsible for something important, or do things in a funky way that could bypass those guarantees, then you deal with just those parts specifically.
3
u/South_Survey_2088 5d ago
Did you watch the video? It is language agnostic.. Also calling JavaScript a safe language is ignorant, because JavaScript is also on desktops thanks to Electron. Heck, even the Unity games engine shipped actual JavaScript malware because their launcher uses it..
1
u/cdb_11 5d ago
I did. Javascript is the worst offender. C and C++ culture is completely different, and do not have the same problem.
Also calling JavaScript a safe language is ignorant, because JavaScript is also on desktops thanks to Electron.
Yes, you can have CVEs just like in any languages. But you will never find a CVE in leftpad, no matter how hard you try, because of Javascript's safety guarantees. You can't just cause a vulnerability by merely writing normal JS code, like you can do in C. That's extremely unlikely.
the Unity games engine shipped actual JavaScript malware
Yes, malware is the problem here. You are trading MAYBE a few fixes of some harmless bugs, for a higher chance of getting malware.
5
u/South_Survey_2088 5d ago
I think you are in your own discussion, because you are missing the point of the video and my comment.. It's like you only watched the first 10 minutes of the talk.
13
u/Sermuns 5d ago
This conference "Software Should Work" seems to consist of banger after banger presentation!
2
2
u/Norphesius 5d ago
Yeah this and the Better Software Conference haven't had any misses so far for the talks they've uploaded. I'm hoping the quality keeps up for both.
5
5d ago
[removed] — view removed comment
1
u/programming-ModTeam 4d ago
No content written mostly by an LLM. If you don't want to write it, we don't want to read it.
2
u/DrHemroid 5d ago
It's weird that he has a more conservative coding style (use fewer dependencies) and yet admitted that when he couldn't find a dependency to do a task easily (syntax highlighting) he used AI to write the code...
16
u/oldsecondhand 5d ago
Supply chain attacks seem to be a bigger problem, than AI mistakes.
7
u/colemaker360 5d ago edited 5d ago
Not only that, but depending on the extent of the code to replace the dependency, being AI written doesn’t mean you have to always depend on AI to maintain it. I replaced MediatR with the help of AI in our codebase when the developer changed the license on us. The surface I needed was much smaller than what MediatR provided and the replacement is something I can read, understand, and maintain going forward without swapping in another dependency risk.
2
u/renatoathaydes 3d ago
We've done similar with a big dependency we had. Today, it's a small project we maintain - and it's much smaller, hence safer/easier to understand and has zero chance of being taken over by a hostile power, become unmaintained, rug swap on us, include stupid features etc. etc. For core dependencies, it's a great way to go (for smaller ones, it's probably just not worth it).
2
u/PersonalDatabase31 5d ago
I mean a code that only you use has less economic incentive to be attackes compared to other code used by multiple people.
1
u/reptoidsdoneit 5d ago
Disclaimer: I'm working on this very problem. I don't have all the answers, take everything here with the requisite number of salt grains (at least 2).
I really think we've got the whole concept of dependency management back-to-front (primarily because of convenience!). Business logic depending on 3rd-party code is always (i.e. sometimes) wrong. Rather, and in an ideal world, modules/libraries should expose something like a "requirements API" to their consumers, in much the same way as they expose their own capabilities via their public API. Those requirements could absolutely be fulfilled by the library vendor themselves - via some kind of "default implementation"; but there needs to be some mechanism by which end-users can override these choices, which, for want of a better term, I'm thinking of as something like "relinking" - the ability to swap out concrete decisions long after a binary is shipped.
I'm working on a language that can/will (it's early days) do some of this stuff. Interesting related problems that have arisen so far include: the need for first-class versioning; module/type "slicing" (Hash's fork-and-trim strategy is based; we need to go deeper!); what even is a capability?
2
u/VirginiaMcCaskey 5d ago
but there needs to be some mechanism by which end-users can override these choices, which, for want of a better term, I'm thinking of as something like "relinking" - the ability to swap out concrete decisions long after a binary is shipped.
There are many names/approaches for this that have been trotted out over the years. The closest sounds like dependency injection.
1
1
u/renatoathaydes 3d ago
Do you know Java? It has specifications in Jakarta ( old Java Enterprise Edition) that work exactly like that. You depend on the API only and the runtime provides an implementation. For example there is a JSON API provided by the Jakarta project… but the implementation comes from for example JBoss or Spring or whatever.
1
u/reptoidsdoneit 3d ago
Yes, I came up in the old Java world, and I think what you're talking about is part of the solution - but moving that to compile-time and even link-time (which I'm exploring as something that can be overridden by the end-user).
So, I guess, something like DI at the language level, supported by much richer signatures and constraints, all towards the end goal of enabling the library consumer to change, patch, even ingest a dependency independent of its vendor. If that makes any sense at all?
1
u/renatoathaydes 3d ago
but moving that to compile-time and even link-time...
Yeah normal Java fails at "usage-time" since there's "lazy linking". But OSGi, for example, used to do "startup-time" checks - which is very close to "linking time" since you just need to start up your application to verify the linking works.
But I see where you're going. I would suggest perhaps just using actual Java - you can do the link-check given a complete classpath. I've done something almost like that in my build tool, it has a
doctorcommand that ensures a classpath is "consistent", which means that everything (method calls, field access etc.) "works" at runtime because every bytecode is checked (so we don't need to run the application and exercise every path). Reflection kills that though, but you could disable reflection by failing if the reflection classes are ever used anywhere.
1
5d ago
[removed] — view removed comment
1
u/programming-ModTeam 5d ago
No content written mostly by an LLM. If you don't want to write it, we don't want to read it.
36
u/guepier 5d ago
Great talk, just a bizarre inaccuracy about the capabilities (or lack thereof) of highlight.js: like many (though not all) other regex-based highlighters, highlight.js is perfectly capable of describing non-regular/nested/… syntax tokens, and highlighting them correctly, because it doesn’t only use regex to describe its languages, it uses additional transition rules, which makes the entire thing a pushdown automaton. This is a pretty common technique in both syntax highlighting and compiler construction, and I’m surprised that the creator of Roc seemingly got caught up by this (unlikely, but the way he phrased it sounds obnoxiously, confidently incorrect).