Are C And C++ Languages Ready For The Npm Debacle
Are the C and C++ languages communities ready for such problem? The short answer is yes. The problem is that we are ready because we don’t have such a dependency on a central repository hosting 99% of the existing OSS libraries we depend on. So we are not there yet.
Conan is there to try to fill this gap. I think this is a good opportunity to explain some of the design decisions about it, and what security and other reasons have affected conan current design:
Allowing removal of packages
Some people argue that packages should not be allowed to be removed, even by the authors. This is only half part of the story, they are concerned only about they role as package consumers but not as creators. This approach might work if you are already Maven Central, but guess what, it doesn’t work for a young project. We already experienced that: package developers want to be able to totally remove their contributions, and they will be very upset if not allowed. And if there are not package creators contributing, there are not packages, and the platform has little value. So, it is absolutely necessary to allow it, otherwise the platform will never bootstrap enough. But this is not a problem per se, if it can be effectively managed.
Yes, we (in C++) love namespaces. They are necessary, vital. Why in heaven should someone just install “string1.0” from a package manager? So packages in conan are referenced by the name and version of the package, but also the author and the channel, something like Boost/1.60@boostorg/stable. So when you depend on it, you are certain about the creator of the package. If the author of the package decides to remove the package, it cannot be replaced by anyone else.
With conan there are two ways to create packages from others:
- If they have a repository with the package recipe, just clone it and export it under your user name. You can generate your own binaries locally or in CI, and upload your own recipe and packages. They are a full copy, it will not be influenced by removal of the original source
- It is also possible to just copy/rename existing packages, including binaries, with the command “conan copy”
- Then, just change the namespace:
Conan is designed to be distributed, so if the origin or availability of packages from external sources is a concern, setting up your own conan server is very simple. Read your dependencies from other origins (as conan.io), then upload them to your own conan server, and you are independent from the outside world.
Package recipes and binaries manifests
Yes, artifacts must have a signature. Now, we generate manifests (a file list with each file SHA) inside each package recipe and each package binary. We use those manifests to resume failed uploads, or to check for upstream updates. As the generators provide the paths for your project dependencies, matching the retrieved manifests against predefined ones shouldn’t be difficult.
conanmanifest.txt of a ZMQ example package
We think that with these measures, conan is quite prepared to avoid such possible problem in the future. This is not a post to claim that we are good, smart, or whatever, just sharing some of our design ideas. Do you have any other ideas? Contributions, suggestions? Please tell us, or even better, enjoy our community of contributors with some pull requests :D