Introducing Conan 2.0
Managing C++ dependencies has traditionally been a challenging problem. Since Conan’s inception, we have strived to deliver a tool that is useful, empowers developers, and saves them time. Since Conan 1.0 was first released, we have witnessed how Conan has become a key part in the C++ ecosystem, giving way to a rich community of developers from all industries. In the past several years, C++ has continued to evolve, and with that, so have the needs of our users.
We have continued to listen to the feedback of this great community, and today we are proud to announce the release of Conan 2.0.
Conan 2.0, the new version of the open-source C and C++ package manager, is out now!
Conan 2.0 has been re-architected from the ground up to better serve the needs of the community. Some notable features:
- New and improved graph model to better represent transitive usage requirements
- Requirement traits
- Package types
- Improved modeling of build-time only dependencies
- Improved support for cross-building workflows
- New documentation, written from scratch
- Hands-on tutorial covering the most common use cases
- Comprehensive and better organized API reference
- Newly structured examples section
- Documented way to deploy Conan without Python in the developers computers
- Revisions for package immutability
- Previously an opt-in in Conan 1.x, revisions are enabled by default in Conan 2.0
- Improved processes for downloading, installing and updating dependencies with a focus on guaranteeing immutability
- More efficient upload and download operations
- Re-Architected and improved build system integrations
- Fully transparent integration with existing CMake projects
- Support for CMake presets to simplify CMake command invocation in local workflows
- Same model across all integrations: Autotools, Xcode, MSBuild and better segregation of logic with the generate() method
- New Command Line interface
- Uniform interface across the multiple Conan commands
- Allow multiple output formats and terminal redirection. Cleaner output.
- Improved, structured machine-readable output (json) to better support CI workflows
- New public Python API
- The same API used by Conan’s built-in commands is now available to use in more advanced integrations
- Custom commands
- Ability for user provided Conan sub-commands, to extend the built-in functionality
- Custom commands can be shared with the well-known
conan config install
mechanism - Can use the new public Python API to implement additional functionality
- Improved package_id logic
- Taking into account package types and traits to more efficiently determine when a package needs to be rebuilt based on changes in its dependencies
- Global package_id compatibility extension
- Ability to define binary compatibility at a global level rather than at the recipe level
- Can be used to exploit known heuristics, for example, that binaries for a given compiler version are compatible and can be linked with binaries built with earlier versions of the same compiler
- Lockfiles: simplified and improved
- Lockfiles are now modeled as list of sorted references, which allow a single lockfile to be used for multiple configurations
- Improved workflows for merging lockfiles, user-defined constraints, and much more
- New configuration system
- Uniform ability to configure and control Conan behavior via profiles and command line via the
[conf]
mechanism - Better structured and flexible than the legacy mix of environment variables.
- Uniform ability to configure and control Conan behavior via profiles and command line via the
- Multi-revision local Conan cache
- Completely redesigned cache to allow storing more than one recipe and package revision at a time.
- Shortened paths, removing the need to workaround path length limitations on Windows
- New deployers
- New ability for user-provided deployment strategies to perform copy operations from the Conan cache to arbitrary user folders.
- Can be used to generate self-consistent prefixes with all dependencies requirements of an application, or to package up dependencies for distribution outside of Conan packages
- New extensibility capabilities
- New ability to customize available settings in
settings_user.yml
without modifying the globalsettings.yml
- Command wrapper: intercept calls to self.run(), useful for external integrations (distributed compilation, etc…)
- Profiles and configuration now support jinja templates for additional flexibility
- New ability to customize available settings in
- Package signing
- New extension to implement signatures and verification of packages
Continue reading to learn more about Conan 2.0 and what it means for your C++ projects.
A little history
The first preview version of Conan was released in late 2016. Since then, Conan has experienced significant continued growth: empowering teams to bring their C++ projects into the DevOps era. In the period between January 2020 and December 2022, we have seen a 5x increase in the utilization of Conan packages in Artifactory (JFrog Software Artifact State of the Union 2023), further signaling an increase in user base that continues to this day.
Thousands of teams are using Conan in their C and C++ workflows, ranging from startups to many large enterprises, including many in the Fortune 100. We have made every effort to keep our promise of stability, continually delivering updates to the Conan 1.x series. But this task has become more difficult over time, as we strive to continue improving upon user feedback, without breaking existing users.
Years of feedback and experience from the community have led to this new milestone, for example:
- In 2022 more than 5000 pull requests were processed in ConanCenter
- +750K downloads/month from PyPI, being designated a PyPI critical project
- Conan 2.0 has closed more than 1000 tickets in Github
- Conan channel in #CppLang with more than 2400 subscribers is consistently ranked among top most active channels
We have been listening to you: today we are proud to announce the release of Conan 2.0.
The needs of C++ projects today
When Conan was first introduced, it happened around the last time big compiler vendors broke the ABI of their runtime libraries (libstdc++ ABI and Visual C++ 2015). Being able to model different binary packages built from the same sources has always been central to Conan’s proposition. Since then, the C++ ecosystem has continued to evolve, faster than ever before:
- Compiler vendors have continued to implement features from the new standards (C++ 14, 17, 20 and beyond), releasing more frequent updates. We have successfully witnessed how teams have adopted Conan to enable developers to adapt to these rapid changes more quickly and robustly than ever before.
- Modern CMake (propagation of transitive usage requirements) has continued to evolve and propagate across the ecosystem. And with this, so have the needs of our users with regards to consuming C++ dependencies in their CMake projects.
- A lot of companies are now developing to target AI/machine learning on embedded devices, and both Apple and Microsoft are now supporting ARM64 as primary targets for their desktop operating systems (macOS and Windows). As such, the needs of our users to support cross-building continues to increase. This is specially the case for IoT applications, as well as robotics, automotive, aviation, embedded and medical industries.
The adoption of Conan by teams of all sizes, but especially the bigger ones, has driven the demand for teams to adopt modern DevOps practices in their C++ projects. Today, we regularly see teams requesting guidance for the following:
- CI integration to build cross-platform C++ projects using Conan’s platform-agnostic abstractions, whereas in the past teams may have been using different dependency solutions on different platforms. How to manage the ever growing combination of platform variations?
- Building dependencies at scale on CI systems on multiple platforms. How to correctly model the minimal dependencies that need to be rebuilt as a result of upstream changes? How to obtain information from a list of a dependency to parallelise and orchestrate package builds across multiple systems?
- How to ensure reproducibility, across CI systems and developers, given that recipes themselves can change over time?
- How to balance the flexibility developers need working on different branches while maintaining the deterministic requirements for CI/CD systems for production software.
During the past 5 years, we have tried to respond to these needs with incremental updates to Conan 1.x. In that process we learned a few important lessons that have motivated us to propose a major 2.0 version to be able to leap to the next level.
Lessons learned
Lesson #1: C and C++ are challenging
Conan 1.0 had a relatively simple dependency model inspired by other language package managers like Python or NPM. Packages could “require” other packages, and as a useful innovation we introduced the concept of a “tool_require” to express dependencies on packages for build tools. But if C and C++ are challenging in themselves, building C and C++ projects is even more challenging, and not surprisingly, we learned that doing package management for C and C++ was also challenging. The simple “require” model was not enough to represent many different scenarios, for example:
- How usage requirements (compiler and linker flags) need to propagate for 2 libraries depending on if they are shared libraries, static libraries, header-only libraries, or a combination of them.
- Whether the visibility of the headers has to be propagated or not to downstream dependents.
- The possibility of the same dependency graph containing multiple versions of the same library - for example, 2 different versions of the same library built as static libraries, but embedded and hidden into 2 different shared libraries.
Creating different new types of “requires”, like “tool_requires” didn’t seem like an idea that could scale, so inspired by how build systems model transitive usage requirements, we created a model in which “requires” can be qualified with several different “traits”, that define how packages are required and how their information is propagated down the dependency graph to other packages depending on them. In this way, one package can define that it requires other package with headers=True, libs=False, transitive_headers=True
, which means it is using only the header files of the library, and it is using them in its own headers, so they will be visible for downstream consumers.
This information can be leveraged in very useful ways, and Conan 2.0 will be way more efficient, fetching only the binaries that it needs to build and run (thanks to a better “package_id” computation model), but being able to skip the download of large parts of the graphs binaries in many cases, accelerating install and build time. We are excited about the potential of the new graph model, the requirements traits and the new package_type recipe attribute, and how this will enable better modeling and management of dependencies.
“With every single release we have seen great improvement to our development chain - we are looking forward to the major leap bringing us to the next level!”
Maikel van den Hurk, Staff Software Engineer, TomTom
Lesson #2. C and C++ are the high performance languages trusted by enterprises worldwide
C and C++ are the backbone of the world’s Software industry. C and C++ are also very commonly the programming language of choice for enterprise-level software. While they may not be part of the tech stack of many smaller companies, they continue to have immense and growing popularity in critical applications like hardware, automotive, finance, medical, AI, media, robotics, embedded, etc. That also means a wild variety of use cases, flows, processes, policies, regulations and compliance rules, etc.
Thus, it is impossible that one tool completely fits such a wide spectrum of use cases, and the extension capabilities are a fundamental part of it. Conan 1.X had limited extensibility, there was a Python API, but it only minimally mimicked the CLI commands, was generally not very useful in practice, and as it was not really designed to be open, as it was never public or documented. But we became painfully aware of this need, because despite the fact that it was undocumented and private, many users have reported hooking into Conan internals.
So we decided that Conan 2.0 would also implement a framework for C and C++ package management, including the possibility of creating your own custom commands, that can be shared and distributed (to developer teams, CI machines, etc), a new more detailed full python API to build those commands, and a new scalable configuration system that allows types (lists, dicts), and high level operations like append/prepend/unset.
Likewise, a new set of extension capabilities have been added to 2.0 to overcome some limitations that we heard about in Conan 1.0. For example, binary compatibility is very challenging, and cannot be defined globally for all cases. A new compatibility.py
plugin that allows users to define their own binary compatibility for their projects has been added. Also, as the industry focus on security keeps increasing, Conan 2.0 has added a new package signing plugin that allows packages to be signed, and a reference implementation with Sigstore will be published after the release.
“Already with Conan 1.x, we were amazed by how well it covers our (sometimes quite special) use cases with just the right mixture of a robust framework and lots of flexibility. As embedded system developers, Conan 2.0 brings us an even better control of our build processes by adding new generators and more fine grained package environments. We’re also looking forward to the new public Python APIs and extension system which will boost our capabilities in C++ build automation.”
Maxime Bergantz, Senior Software Engineer at Bosch
Lesson #3: C++ DevOps at Scale
Since 2016, as Conan usage increased, the size of the problems also started to increase. Being able to manage C++ dependencies, enabled companies to do more things, more quickly, and that started the desire for going even further. We then learned that we were entering the world of DevOps for C++, something that other technologies have been doing for some time. The importance for some users to be able to accurately reproduce a dependency graph several years after that graph was used to build a certain product version. In order to address this need, Conan introduced lockfiles. These lockfiles became relevant for trying to distribute the build of a large dependency graph in a CI cluster with as much parallelism as possible.
However, this important tool was complicated to manage, and implementing CI and DevOps at scale was quite challenging, becoming a bottleneck for many companies to be able to accelerate more in their delivery. Unfortunately, we also painfully realized that it was not possible to fix that bottleneck in Conan 1.X scope without breaking.
So we are finally very excited to launch 2.0, with new simpler and more powerful lockfiles, and new improved CI oriented tools, like a better computation of the build-order of packages in a graph, and a better optimization of what has to be built in a dependency graph when something changes in such graph. With these new tools, it will finally be possible to unlock the full potential of modern devops processes for C and C++ projects.
“I am really excited for Conan 2.0. By starting the Conan tribe in 2020, the Conan team has made sure to incorporate user feedback into this upcoming release. I’m really looking forward to the improved lockfile handling which Conan 2.0 will bring. Together with the new Python API, this will greatly simplify our CI workflows.”
Kerstin Keller, Software Developer, Continental
A thank you to the community
It should go without saying that none of this would be possible without the support of the Conan community. We have always prided ourselves on listening to the user base, and it’s one of the most challenging aspects of developing a tool like this. Sometimes what’s best for the vast majority has an undesired taste for a few individuals. We sometimes need to make tough decisions about which features to prioritize or which auxiliary projects to support - but through it all we have always tried our best to keep even the smallest voices in mind.
To our enterprise class users from the Conan Tribe to the individuals who ask questions on GitHub issues, who’ve helped inspire the changes and new features we’ve brought in Conan 2.0, thank you.
To the heroes contributing and maintaining the ~1500 recipes that are now in ConanCenter - we raise a toast. Conan Center enables newcomers to try Conan out more easily, all the while being a showcase for how to use Conan at scale in enterprise environments. We hope you join us in the celebration of this launch.
To the contributors who have opened issues, reported bugs, or shared real use cases - we appreciate your feedback. It has helped in no small part in bringing Conan to the current level and we look forward to hearing from you on this journey with us.
To the C++ ecosystem that has welcomed us to conferences and meetups, including Conan in annual surveys and beyond. We look forward to mingling again soon.
None of this would be possible without the support of JFrog. Open-Source projects might be free for users but the development and support take a lot of resources. Thank you for helping make C++ package management a reality.
Get started with Conan 2.0 today
Conan 2.0 is now available for download. To get started, head to our downloads section and get started with the Conan 2.0 documentation.
Note about ConanCenter
In the months leading up to today’s launch, community contributors and the Conan team have made great efforts to upgrade many of the ConanCenter recipes to be compatible with Conan 2.0, as well as retaining compatibility with Conan 1.X. At launch time, the top 100 most downloaded recipes have been validated to be compatible with Conan 2.0.
The efforts will continue in the following weeks and months to upgrade as many Conan Center recipes to Conan 2.0 as possible - To read more about the progress of this task, read the Conan Center announcement.