-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathhunterintro.html
252 lines (227 loc) · 18.3 KB
/
hunterintro.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
<!DOCTYPE html>
<html lang="en">
<head>
<!-- 2019-07-25 Thu 11:01 -->
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>A Quick introduction to Hunter</title>
<meta name="generator" content="Org mode">
<meta name="author" content="George Kontsevich">
<meta name="description" content="Notes from setting up things on Linux"
>
<link rel="stylesheet" type="text/css" href="../web/worg.css" />
<link rel="shortcut icon" href="../web/panda.svg" type="image/x-icon">
</head>
<body>
<div id="org-div-home-and-up">
<a accesskey="h" href=".."> UP </a>
|
<a accesskey="H" href=".."> HOME </a>
</div><div id="content">
<h1 class="title">A Quick introduction to Hunter</h1>
<div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#org77e3b91">Motivation</a>
<ul>
<li><a href="#org0fbe08d">Alternate dependency managment strategies</a>
<ul>
<li><a href="#org8ddc5e9">ExternalProject<sub>Add</sub></a></li>
<li><a href="#orge81d7bd">git-submodule</a></li>
<li><a href="#org5c88d9c">a bigger pervasive issue.. DRY</a></li>
</ul>
</li>
<li><a href="#org6368926">The Hunter fixes</a></li>
</ul>
</li>
<li><a href="#org02cb645">Integrating Hunter</a></li>
<li><a href="#org1ef899c">Adding dependencies</a></li>
<li><a href="#org8849a39">Customizing dependencies</a></li>
<li><a href="#org3a10bae">Toolchain files</a></li>
<li><a href="#orgff3626e">Files living on your system</a></li>
<li><a href="#org5c369eb">Adding unsupported projects</a></li>
</ul>
</div>
</div>
<div id="outline-container-org77e3b91" class="outline-2">
<h2 id="org77e3b91">Motivation</h2>
<div class="outline-text-2" id="text-org77e3b91">
<p>
<code>Hunter</code> is a easy-to-use and easy-to-integrate dependency manager for C++ projects that run <code>CMake</code>. It works entirely from within <code>CMake</code>, so you don't need any extra software. Once your project is integrated with <code>Hunter</code>, contributors can clone your repository and immediately start working. During <code>CMake</code>'s configuration step all dependencies will be downloaded and built from source. This makes your project 100% independent of system libraries (other than the system's standard-library/libc)
</p>
</div>
<div id="outline-container-org0fbe08d" class="outline-3">
<h3 id="org0fbe08d">Alternate dependency managment strategies</h3>
<div class="outline-text-3" id="text-org0fbe08d">
</div>
<div id="outline-container-org8ddc5e9" class="outline-4">
<h4 id="org8ddc5e9">ExternalProject<sub>Add</sub></h4>
<div class="outline-text-4" id="text-org8ddc5e9">
<p>
<a href="https://cmake.org/cmake/help/v3.2/module/ExternalProject.html">ExternalProject<sub>Add</sub>()</a> is a functionality built into <code>CMake</code> itself and will build 3rd party projects <i>in parallel</i> to your own project. It allows for custom build/install instructions and can support non-CMake based dependencies. It will fetch source code from <code>URL</code>'s / <code>git</code> and is very flexible but it has some serious drawbacks. Libraries are added by using <code>find_package()</code> and then juggling the <code>CMake</code> variables you get back.
</p>
<p>
The reality is that it only works out-of-the-box for header-only libraries. Because everything is built at once, if you need to link to a library depedency, the lib won't actually exit during the configuration step so your configuration will fail. (the file will only be created after the build step) You can hack <code>CMake</code> so that you configure and build your dependencies first, and then configure and build your parent project - but for some reason <code>CMake</code> doesn't support this normally. Almost every extra dependency requires a tweaked hack and the solution is messy and hard to maintain.
</p>
</div>
</div>
<div id="outline-container-orge81d7bd" class="outline-4">
<h4 id="orge81d7bd">git-submodule</h4>
<div class="outline-text-4" id="text-orge81d7bd">
<p>
A better solution (for dependencies that use <code>CMake</code>) is to put them in a <code>git-submodule</code>. This is described in more detail <a href="http://foonathan.net/blog/2016/07/07/cmake-dependency-handling.html">in this blog</a>, but typically your dependencies will live in a folder like <code>/contrib/</code> or <code>/externals/</code> and are directly linked to other repositories and don't bloat your own codebase. The submodule can be pinned to a commit or it can track a branch/tag so you get the latest changes. To use the submodule you simply add its top-level <code>CMakeLists.txt</code> directory with <code>add_subdirectory()</code> - just like it was another folder in your project - and then the submodule's targets become available directly for you to link with. This is a very clean solution because:
</p>
<p>
<b>1</b>: It allows you to never worry about include directories (when you use <code>target_link_libraries()</code> to link a target library <code>CMake</code> will automatically include not only its own public headers but also headers of secondary dependencies when necessary)
</p>
<p>
<b>2</b>: Not have to deal with <code>CMake</code> variables or directory paths which you would get using <code>find_package()</code>.
</p>
<p>
But in a large project you will quickly hit some issues:
</p>
<p>
<b>1</b>: Many dependencies are not setup to be used as <code>CMake</code> subdirectories. Variables such as the project-version-number get overwritten by the parent project and more dangerously the submodule can start to change variables in the overall project. Best case your configuration will crash, worst case you will start to get strange behavior that will go unnoticed.
</p>
<p>
<b>2</b>: Dependencies will often reuse target names. For instance, several popular libraries have a target called <code>uninstall</code>. Because redeclaring a target in a different subdirectory is a no-no and <code>CMake</code> will crash.
</p>
</div>
</div>
<div id="outline-container-org5c88d9c" class="outline-4">
<h4 id="org5c88d9c">a bigger pervasive issue.. DRY</h4>
<div class="outline-text-4" id="text-org5c88d9c">
<p>
The last issue affect both solutions and has no adequate fix in vanilla <code>CMake</code>. Mutliple libraries will share dependencies, especially with common libraries like <code>zlib</code>. Sometimes you will see a library simply build its own copy of <code>zlib</code> without asking, sometimes you can override it - but what you end up with is a final binary that has multiple linked <code>zlib</code>'s - typically at different version numbers! This <i>usually</i> works fine, but is undefined behavior. The linker doesn't have mechanisms for managing this. Not only is this sloppy, but the more libraries you have the more this becomes a potential issue.
</p>
</div>
</div>
</div>
<div id="outline-container-org6368926" class="outline-3">
<h3 id="org6368926">The Hunter fixes</h3>
<div class="outline-text-3" id="text-org6368926">
<p>
<b>Dependencies are built at configuration time</b>:
Hunter will download the source and build all your dependencies during configuration time.
</p>
<p>
<b>Dependencies are built independently</b>:
Each dependency is configured/built/installed independently and the only thing they inherit from the parent project is the <i>toolchain file</i> you passed in at the command line. (the <i>source</i>, <i>build</i> and <i>install</i> directories are in a hidden folder which you never have to touch and won't affect the rest of your system)
</p>
<p>
<b>Dependencies export their targets in namespaces</b>:
Even though dependency targets are installed at configuration time, they still get exported to the parent project. So you do not have to revert to the usual post-install <code>find_package()</code> way of dealing with <code>CMake</code> variables and include paths. All the dependency targets remain available to the parent project but are namespaced in a PROJECT::TARGET pattern to avoid the target name collisions we had in the <code>git-submodule</code> case. So for instance Boost's target <code>filesystem</code> will now be <code>Boost::filesystem</code>.
</p>
<p>
<b>Dependencies share their own dependencies</b>:
If two dependencies need <code>zlib</code> then <code>Hunter</code> will install just one <code>zlib</code> and link both dependencies to it
</p>
<blockquote>
<p>
<b>Note</b>: This is a good time to mention a major draw-back of using <code>Hunter</code>. As you can probably guess at this point, making a dependency work with <code>Hunter</code> requires making some changes that tell <code>Hunter</code> what libraries it needs. Fortunately <i>Hunterized</i> versions of most commonly used libraries are available on <a href="https://docs.hunter.sh/en/latest/packages/all.html">their website</a>
</p>
</blockquote>
</div>
</div>
</div>
<div id="outline-container-org02cb645" class="outline-2">
<h2 id="org02cb645">Integrating Hunter</h2>
<div class="outline-text-2" id="text-org02cb645">
<p>
Adding <code>Hunter</code> to an existing project is actually pretty easy. <code>Hunter</code> is split into two parts. A small gate file which seems to define all <code>Hunter</code>'s <code>CMake</code> functions, and the <code>Hunter</code> archive which holds all the information about the dependencies (hashes,urls,version-numbers,etc.). Copy the gate file <code>/cmake/HunterGate.cmake</code> from <a href="https://github.com/hunter-packages/gate">their repository</a> into your own project directory and then you can load <code>Hunter</code> and its archive in 2 lines at the top of your <code>CMakeLists.txt</code> (be sure it's before the <code>project(ProjectName)</code> line):
</p>
<div class="org-src-container">
<pre class="src src-cmake"><span style="color: #006699;">cmake_minimum_required</span>(VERSION 3.1)
<span style="color: #8D8D84; font-style: italic;"># HUNTER dependencency manager</span>
<span style="color: #006699;">include</span>(<span style="color: #008000;">"cmake/HunterGate.cmake"</span>)
<span style="color: #006699;">HunterGate</span>(
URL <span style="color: #008000;">"https://github.com/ruslo/hunter/archive/v0.19.240.tar.gz"</span>
SHA1 <span style="color: #008000;">"aa3cd9c45391d8bd14441971b00a43c05d40347c"</span>
LOCAL
)
<span style="color: #006699;">project</span>(ExampleProject)
</pre>
</div>
<p>
The third line here loads a particular version of the archive so you don't have to worry about the archive being change from under you. More recent versions are available at:<a href="https://github.com/ruslo/hunter/releases">https://github.com/ruslo/hunter/releases</a> and as you can see they are updated every few day as new versions of different libraries come out. The gate file in contrast is pretty stable.
</p>
</div>
</div>
<div id="outline-container-org1ef899c" class="outline-2">
<h2 id="org1ef899c">Adding dependencies</h2>
<div class="outline-text-2" id="text-org1ef899c">
<p>
Now that we have <code>Hunter</code> in our project we want to add dependencies. This turns out to be a quick ttwo lines of code. Before <code>Hunter</code> we'd look for system libraries and then get the resulting variable to link it in to our code
</p>
<div class="org-src-container">
<pre class="src src-cmake"><span style="color: #8D8D84; font-style: italic;"># CMAKE STYLE</span>
<span style="color: #006699;">find_package</span>(Boost)
...
<span style="color: #006699;">target_link_libraries</span>(OurTarget ${<span style="color: #BA36A5;">Boost_Lib_Pack</span>})
</pre>
</div>
<p>
Now you just tell <code>Hunter</code> to get <code>Boost</code> with the components we want and we link directly to the targets
</p>
<div class="org-src-container">
<pre class="src src-cmake"><span style="color: #8D8D84; font-style: italic;"># HUNTER STYLE</span>
<span style="color: #006699;">hunter_add_package</span>(Boost COMPONENTS chrono date_time filesystem regex system thread)
<span style="color: #006699;">find_package</span>(Boost CONFIG REQUIRED chrono date_time filesystem regex system thread)
...
<span style="color: #006699;">target_link_libraries</span>(OurTarget Boost::chrono Boost::date_time Boost::filesystem Boost::regex Boost::system Boost::thread)
</pre>
</div>
<p>
And that's it! When you run <code>cmake</code> on your project, <code>Boost</code> will get downloaded on to your system and built. We get to tell <code>CMake</code> which parts we want to link to explicitly and we now only deal with targets and no messy <code>CMake</code> variables.
</p>
</div>
</div>
<div id="outline-container-org8849a39" class="outline-2">
<h2 id="org8849a39">Customizing dependencies</h2>
<div class="outline-text-2" id="text-org8849a39">
<p>
So far everything is working great and it's about on-par with linking to system libraries. The build is repeatable and should work on any system you throw it on. But it would be nice to have a bit more control over the dependencies. We still haven't really addressed what version are the dependencies being built at and what build flags they use. Often libraries, like <code>OpenCV</code> and <code>Boost</code> will have dozens of components and features that we can toggled. These are all controlled with CMake variables that get passed in during configuration, either on the command line <code>cmake -DOPTION1=ON/OFF</code> or in a cmake GUI. When you use a <code>git-submodule</code> you can declare these variables in the parent project and they would toggle things in the subdirectory, but as I said before, <code>Hunter</code> builds things separately and doesn't pass anything between parent project and dependency (except for the toolchain file). So <code>Hunter</code> has a separate dependency configuration file <code>/cmake/Hunter/config.cmake</code> (it's optional). This file simply has a list of lines reading <code>hunter_config({dependency name} {version number} CMAKE_ARGS {...list of cmake args you would pass in at the command line...)</code>. Some examples:
</p>
<div class="org-src-container">
<pre class="src src-cmake"><span style="color: #006699;">hunter_config</span>(OpenCV VERSION 3.3.1-p1 CMAKE_ARGS BUILD_DOCS=OFF BUILD_EXAMPLES=OFF WITH_IPP=OFF WITH_IPP_A=OFF)
<span style="color: #006699;">hunter_config</span>(CURL VERSION 7.49.1-DEV-v9 CMAKE_ARGS CMAKE_USE_OPENSSL=ON CMAKE_USE_LIBSSH2=OFF BUILD_CURL_EXE=OFF BUILD_CURL_TESTS=OFF)
</pre>
</div>
<p>
The versions can either be found on the <i>hunterized</i> repository or more conveniently in <a href="https://github.com/ruslo/hunter/tree/master/cmake/projects">the archive itself</a>. For instance if we look at the <code>OpenCV</code> archive definition file: <a href="https://github.com/ruslo/hunter/blob/master/cmake/projects/OpenCV/hunter.cmake">https://github.com/ruslo/hunter/blob/master/cmake/projects/OpenCV/hunter.cmake</a> We can see multiple versions that have the <code>p</code> suffix (which I think indicates a hunterized branch of the original).
</p>
</div>
</div>
<div id="outline-container-org3a10bae" class="outline-2">
<h2 id="org3a10bae">Toolchain files</h2>
<div class="outline-text-2" id="text-org3a10bae">
<p>
There is one caveat to configuration - often project will provide flags for selecting if you want to make a shared library, or a static one (and other things of that sort). I would strongly urge not playing with these setting. Everything that can be specified in the toolchain file should be in the toolchain file b/c as mentioned a couple of times, the only part that is passed from your parent project to the dependencies is the toolchain file. That means that your project and all it's dependencies will be built with the same exact tools and with the same build flags. If you start forcing contradictory things in your configurations, I'm not really sure what the result will be. Keeping everything in the toolchain file is great for consistency and for trying out different configurations for your whole "stack". If you want to rebuild everything with debug information it's a simple toolchain file swap away. If you want to build for a different platform, compiler, strip all the symbols, etc. all can be quickly done with a change of a toolchain. Because toolchain files are a bit tricky to write, the <code>Hunter</code> maintainer has a separate project that maintains toolchains called <a href="https://github.com/ruslo/polly">Polly</a>. I recommend cloning the repository and looking at the different toolchains files. They're organized in a very modular way which makes modifying and creating your own toolchains less scary and almost painless.
</p>
</div>
</div>
<div id="outline-container-orgff3626e" class="outline-2">
<h2 id="orgff3626e">Files living on your system</h2>
<div class="outline-text-2" id="text-orgff3626e">
<p>
The last big question that arises is, for instance, if you build with debug information turned on and then switch to building release, is all your debug build information lost? Will you have to rebuild everything again when you build debug again? The answer is no. All dependency source/build/installs are cached in a special central system directory (on Linux it's in <code>/home/username/.hunter/</code> and on Windows you need to set it manually). They're saved with a hash based on the <i>version number</i>, <i>configuration flags</i> and <i>toolchain</i> used. So as long as you use the same version/configuration/toolchain on a dependency the build will get reused. However if you play around with a lot of flags you may quickly find your disk space disappearing! So it can be good to periodically delete the directory and just start the cache from scratch.
</p>
</div>
</div>
<div id="outline-container-org5c369eb" class="outline-2">
<h2 id="org5c369eb">Adding unsupported projects</h2>
<div class="outline-text-2" id="text-org5c369eb">
<p>
While the <code>Hunter</code> has pretty much every commonly used dependency - sometimes you will need to add something else. This part I don't really have a good grasp of and the process of forking the <code>Hunter</code> archive and adding a dependency is a bit confusing to me. If you have control over the dependency (it's another library you are working on or are comfortable forking and maintaining) then I suggest ensuring the target names don't collide and just using it as subdirectory. As a subdirectory you can still use <code>Hunter</code> inside of it to manage secondary dependencies so you will still get the benefit of avoid double-linking issues.
</p>
</div>
</div>
</div>
<div id="postamble" class="status">
<p class="author">Author: George Kontsevich</p>
<p class="date">Created: 2019-07-25 Thu 11:01</p>
<p class="validation"><a href="http://validator.w3.org/check?uri=referer">Validate</a></p>
</div>
</body>
</html>