Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update/fix docs #41

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

Update/fix docs #41

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b58598d
Update/fix packaging doc
JedMeister Dec 13, 2024
2cfb30e
Tweak readme
JedMeister Feb 19, 2025
79675b8
Fix 'building packages from source' link
JedMeister Feb 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 12 additions & 9 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,20 +1,21 @@
TKLDev - TurnKey Development Toolchain and Build System
=======================================================

TKLDev is the mother of all TurnKey apps. It's used to give birth to all
TurnKey apps, including new versions of `itself`_. It's designed to make
TKLDev_ is the mother of all TurnKey appliances. It's used to give birth
to them all, including new versions of itself. It's designed to make
simple things simple, and hard things possible. It's a self-contained
build system that can be used to rapidly prototype and repeatably build
any generic Debian-based Linux distribution or TurnKey GNU/Linux system
`from source`_.
TurnKey appliances `from source`_. Or for that matter, any generic
Debian-based Linux distribution. It also supports `building packages
from source`_ too.

To get started read the `documentation`_. Read the `blog post`_
To get started have a look at the `documentation`_. Read the `blog post`_
announcing TKLDev for the background story.

This appliance includes all the standard features in `TurnKey Core`_,
and on top of that:

- Includes a fast copy-on-write build toolchain based on
- Includes a fast copy-on-write build toolchain based on
GNU make, `fab`_, and `deck`_.
- Turnkey development directory structure with preconfigured CDPATH.
- Integrated caching proxy (squid) for fast and iterative development.
Expand All @@ -31,10 +32,12 @@ Credentials *(passwords set at first boot)*

- SSH: username **root**

.. _itself: https://github.com/turnkeylinux-apps/tkldev
.. _TKLDev: https://github.com/turnkeylinux-apps/tkldev
.. _from source: https://github.com/turnkeylinux-apps/
.. _building packages from source: https://github.com/turnkeylinux-apps/tkldev/blob/master/docs/advanced/building-packages-from-source.rst
.. _documentation: https://github.com/turnkeylinux-apps/tkldev/blob/master/docs
.. _blog post: https://www.turnkeylinux.org/blog/introducing-tkldev
.. _TurnKey Core: https://www.turnkeylinux.org/core
.. _fab: https://github.com/turnkeylinux/fab
.. _deck: https://github.com/turnkeylinux/deck
.. _from source: https://github.com/turnkeylinux-apps/
.. _TurnKey Core: https://www.turnkeylinux.org/core
.. _documentation: https://github.com/turnkeylinux-apps/tkldev/tree/master/docs
188 changes: 43 additions & 145 deletions docs/advanced/building-packages-from-source.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,184 +1,82 @@
Building Packages from Source
=============================

As of v14.2, TKLDev includes all the components needed to build packages from
souce code on the fly. This may be useful if you wish to test a piece of
software which is unpackaged, but you want it packaged. It can also be useful
if you wish to tweak an existing TurnKey package.
TKLDev includes most of the components needed to build Debian packages from
souce code OOTB. This can be useful if you want to test changes you've made
to an existing package or create a new package - of your code or someone
else's.

Please note that within these instructions, I make use of variables. If you
follow this from start to finish in a single shell, then you won't have any
issues. Please note though, that if you use multiple shells, or do it over
multiple sessions, then you'll either need to redefine the variables, or
substitue the real path/value.


Building packages - getting set up
----------------------------------

Before you can build packages, you'll need a buildroot to build them in. A
buildroot is essentially a minimalist Debian chroot_. TurnKey may provide one
for download at some point. But for now, you'll need to build it yourself.


Chanko - initial setup
----------------------

Chanko_ is a tool to download and cache Debian packages and is a prerequisite
for building our buildroot.

Initially, you will need to ensure that Chanko is fully configured and has
been pre-loaded with the packages required for the buildroot. If you haven't
already done it, please set up Chanko:

.. code-block:: bash

tkldev-chanko-setup

Note: this script may not yet be included in TKLDev, you may need to download
and prepare the script first:

.. code-block:: bash

GH_URL=https://raw.githubusercontent.com/turnkeylinux-apps/tkldev/master/overlay
SCRIPT=usr/local/bin/tkldev-chanko-setup
wget -O /$SCRIPT $GH_URL/$SCRIPT
chmod +x /$SCRIPT


Loading Chanko
--------------

To load Chanko with the required packages, you'll need to create some plans
which will note the required packages Chanko will need to download.

If you haven't already, please clone the buildroot source code:

.. code-block:: bash

cd /turnkey
git clone https://github.com/turnkeylinux/buildroot.git

Now to gather the plans for Chanko:

.. code-block:: bash

CODENAME=$(lsb_release -cs)
CHANKO=/turnkey/fab/chankos/${CODENAME}.chanko

wget -O ${CHANKO}/plan/bootstrap-required \
https://raw.githubusercontent.com/turnkeylinux/bootstrap/master/plan/required
wget -O ${CHANKO}/plan/bootstrap-base \
https://raw.githubusercontent.com/turnkeylinux/bootstrap/master/plan/base
echo "/* buildroot */" >> ${CHANKO}/plan/buildroot
cat /turnkey/buildroot/plan/main >> ${CHANKO}/plan/buildroot
# extra TurnKey packages for buildroot:
cat >> ${CHANKO}/plan/buildroot <<EOF
turnkey-pylib
pyproject-common
autoversion
EOF
# note the 'EOF' must be at the start of a line and the only thing on said line

Then download the packages like this:
Environment variables
---------------------

.. code-block:: bash

cd ${CHANKO}
chanko refresh -a
echo y | chanko-get plan/bootstrap-base
echo y | chanko-get plan/bootstrap-required
echo y | chanko-get plan/buildroot
# assuming building for the current TurnKey/Debian version
CODENAME=$(lsb_release -sc)

That will take a little while to do it's work, but hopefully not too long.
Buildroot
---------

A buildroot is a minimalist Debian chroot_ specifically for building packages
in.

Build the buildroot
-------------------
TurnKey uses our bootstrap_ as a base, then adds some extra build related
tools/packages. To generate one yourself, see the buildroot_ repo.

Once you've configured Chanko, it's really easy! You just need to do this:

.. code-block:: bash

CODENAME=$(lsb_release -cs)

cd /turnkey/buildroot
export FAB_POOL=y
export RELEASE=debian/$CODENAME
make

Pool configuration
------------------

Put the buildroot in place
--------------------------

Once that has finished, you just need to move your buildroot to where Pool is
expecting it to be:

.. code-block:: bash
Pool_ is the custom TurnKey tool for building packages. With the exception
of a buildroot, TKLDev should have pool configured out of the box. So once
you have built your buildroot you should be good to go.

CODENAME=$(lsb_release -cs)
BUILDROOT=${FAB_PATH}/buildroots/${CODENAME}
mkdir -p ${BUILDROOT}
Build a package with pool
-------------------------

rsync --progress --delete -Hac build/root.patched/ ${BUILDROOT}
I'll use TurnKey's "inithooks" package as an example:

Once that's done, you can recover a bit of space by cleaning up:
Clone or copy your source code to an appropriate location:

.. code-block:: bash

make clean

Initialize Pool
---------------
mkdir /turnkey/public
cd /turnkey/public
git clone https://github.com/turnkeylinux/inithooks.git

The last step is to initialize Pool_.
Then register the source code with Pool and build the package:

.. code-block:: bash

cd pools/
mkdir -p ${CODENAME}
cd ${CODENAME}
pool-init ${FAB_PATH}/buildroots/${CODENAME}

# bugfix
mkdir -p /turnkey/fab/deckdebuilds/chroots/

Now we should be nearly ready to build!

cd pools/${CODENAME}

Build a package from source
---------------------------
pool-register /turnkey/public/inithooks
pool-get . inithooks

Now it's time to build a package! For this test I'm going to build TurnKey's
inithooks package. So let's clone the source code first:
Assuming that the build process completes successfully, you will find the new
Debian package in the pool directory:

.. code-block:: bash

mkdir /turnkey/public
cd /turnkey/public
git clone https://github.com/turnkeylinux/inithooks.git
root@tkldev pools/bookworm# ls
inithooks_2.2.1_all.deb

Now we'll register this source code with Pool and build the package:
If you wish, the package can be installed on the local system using apt:

.. code-block:: bash

cd pools/${CODENAME}

pool-register /turnkey/public/inithooks
pool-get . inithooks
apt install ./inithooks_2.2.1_all.deb

And after a few moments, you should find the ccurl Debian package ready for use:
Alternatively, the package can be used when building an appliance by adding the
package name to the appliance plan (if required) and setting `FAB_POOL=y` prior
to appliance build. E.g.:

.. code-block:: bash

root@tkldev pools/stretch# ls
inithooks_0.9+270+g179b423_all.deb

You can now move this .deb to where you wish to use it and install with dpkg.
Alternatively, if you are building an appliance you can automatically include
this package by adding the packagename to the plan and building with
`export FAB_POOL=y`.
export FAB_POOL=y
make

.. _chroot: https://en.wikipedia.org/wiki/Chroot
.. _Chanko: https://github.com/turnkeylinux/chanko
.. _bootstrap: https://github.com/turnkeylinux/bootstrap
.. _buildroot: https://github.com/turnkeylinux/buildroot
.. _Pool: https://github.com/turnkeylinux/pool