path: root/src/man/poudriere-bulk.8

.\" Copyright (c) 2012 Baptiste Daroussin <bapt@FreeBSD.org>
.\" Copyright (c) 2012-2014 Bryan Drewery <bdrewery@FreeBSD.org>
.\" Copyright (c) 2018 SRI International
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
.\" ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
.\" SUCH DAMAGE.
.\"
.\" $FreeBSD$
.\"
.\" Note: The date here should be updated whenever a non-trivial
.\" change is made to the manual page.
.Dd September 9, 2021
.Dt POUDRIERE-BULK 8
.Os
.Sh NAME
.Nm "poudriere bulk"
.Nd build a ready-to-export package tree
.Sh SYNOPSIS
.Nm
.Fl a
.Fl j Ar name
.Op Fl CcFIikNnRrSTtvw
.Op Fl b Ar name
.Op Fl B Ar name
.Op Fl J Ar maxjobs Ns Op Cm \&: Ns Ar prebuildmaxjobs
.Op Fl O Ar overlay Op Oo Fl O Ar overlay2 Oc Ar ...
.Op Fl p Ar tree
.Op Fl z Ar set
.Nm
.Fl f Ar file Op Oo Fl f Ar file2 Oc Ar ...
.Fl j Ar name
.Op Fl CcFIikNnRrSTtvw
.Op Fl b Ar name
.Op Fl B Ar name
.Op Fl J Ar maxjobs Ns Op Cm \&: Ns Ar prebuildmaxjobs
.Op Fl O Ar overlay Op Oo Fl O Ar overlay2 Oc Ar ...
.Op Fl p Ar tree
.Op Fl z Ar set
.Nm
.Fl j Ar name
.Op Fl CcFIikNnRrSTtvw
.Op Fl b Ar name
.Op Fl B Ar name
.Op Fl J Ar maxjobs Ns Op Cm \&: Ns Ar prebuildmaxjobs
.Op Fl O Ar overlay Op Oo Fl O Ar overlay2 Oc Ar ...
.Op Fl p Ar tree
.Op Fl z Ar set
.Ar origin Op Ar origin2 ...
.Sh DESCRIPTION
This command makes a ready-to-export package tree, and fills it with
binary packages built from a given list of ports.
During the build, hit
.Ic ^T
to send
.Dv SIGINFO
and show stats and progress about the build.
.Pp
See the
.Em FLAVORS
section
in
.Xr poudriere 8
for supported flavors syntax.
.Pp
See
the
.Em CUSTOMIZATION
section
in
.Xr poudriere 8
to learn how to build binary packages with options that differ from
defaults.
.Pp
.Nm poudriere
will disable make jobs for packages not listed in
.Sy ALLOW_MAKE_JOBS_PACKAGES ,
unless
.Sy ALLOW_MAKE_JOBS
is set in
.Pa poudriere.conf .
The number of jobs can be controlled in
.Pa make.conf
with the
.Xr ports 7
flags
.Sy MAKE_JOBS_NUMBER
and
.Sy MAKE_JOBS_NUMBER_LIMIT .
Beware that there is no global job control so each builder created from
.Fl J
(or the default
.Sy PARALLEL_JOBS )
will spawn as many jobs as the ports framework allows.
.Sh SUBCOMMANDS
.Bl -tag -width "-f file"
.It Fl a
Build all ports in the tree with all flavors.
.It Fl f Ar file
Build ports listed in the
.Ar file .
.Pp
The path to the
.Ar file
has to be absolute.
Ports must be specified in the form of
.Dq Ar category Ns / Ns Ar port
and
.Xr sh 1 Ns -style
comments are allowed.
Multiple
.Fl f Ar file
arguments may be specified at once.
.El
.Sh OPTIONS
.Bl -tag -width "-B name"
.It Fl B Ar name
Specify which buildname to use.
By default
.Ar YYYY-MM-DD_HH:MM:SS
will be used.
This can be used to resume a previous build and use the same log and URL paths.
Resuming a build will not retry built/failed/skipped/ignored packages.
.It Fl b Ar name
Specify the
.Ar name
of the binary package branch to use to prefetch packages.
Should be
.Qq latest ,
.Qq quarterly ,
.Qq release_* ,
or
.Ar url .
With this option poudriere will first try to fetch from the binary package
repository specified the binary package prior to do the sanity check if the
package does not already exist.
.Pp
.Nm poudriere
will only use packages that:
.Bl -bullet -compact
.It
come from a repository having the same or older version of
.Sy pkg .
.It
do not have a locally fetched package already.
.It
are not IGNORED.
.It
match the expected local version.
.It
match the expected ABI.
.It
match the expected runtime and library dependencies.
.It
match the expected OPTIONS when
.Cm CHECK_CHANGED_OPTIONS
is enabled (default: on).
.It
The package is NOT listed in
.Cm PACKAGE_FETCH_BLACKLIST
.It
The package is NOT listed with
.Fl C ,
or
.Fl c ,
when
.Fl t
is used.
.El
The
.Fl v
flag can be used to show these decisions during build.
Specifing twice will show more details on why some are skipped.
.Pp
!!
.Pp
.Nm poudriere
has no way of determining differences outside of the above list.
That is, if the local ports framework, or port, has custom patches or special
.Sy WITH_FOO
knobs (not OPTIONS) then it is required to add its name into
.Sy PACKAGE_FETCH_BLACKLIST .
Otherwise a package may be fetched and used that lacks the custom patch or knob.
.Pp
!!
.Pp
See
.Sy PACKAGE_FETCH_BRANCH ,
.Sy PACKAGE_FETCH_URL ,
.Sy PACKAGE_FETCH_BLACKLIST ,
and
.Sy PACKAGE_FETCH_WHITELIST
in
.Pa poudriere.conf.sample .
The entries in the lists will be matched against package names without versions.
.It Fl C
Clean only the packages specified on the command line or in the file given by
.Fl f Ar file .
Implies
.Fl c
for
.Fl a .
.It Fl c
Clean
.Em all
previously built packages and logs.
.It Fl F
Fetch only from the original
.Va MASTER_SITES .
Skip
.Fx
mirrors.
.It Fl I
Advanced interactive mode.
.Pp
Leave the jail running with ports installed after building and testing.
It is a convenient way to do some additional testing.
See
.Xr poudriere-jail 8
to learn how to stop a running poudriere jail.
.Pp
Similarly to
.Fl i ,
the
.Fl I
flag
will cause
.Xr pkg.conf 5
to be installed in the jail.
.It Fl i
Interactive mode.
.Pp
Open an interactive shell session in the jail after the build is done and before the clean-up.
It is a convenient way to do some additional testing.
.Pp
A local
.Xr pkg.conf 5
repository configuration will be installed to
.No ${ Ns Va LOCALBASE Ns } Ns Pa /etc/pkg/repos/local.conf
so that
.Xr pkg 8
can be used with any existing packages built for the jail.
The
default
.Fx
repository will be disabled by default.
.It Fl J Ar maxjobs Ns Op Cm \&: Ns Ar prebuildmaxjobs
Specify the number of jobs that will run in parallel for a bulk build.
The optional second parameter,
.Ar prebuildmaxjobs ,
is the number of jobs used for the steps before the build, they are more IO
bound than CPU bound, so you may want to use a different number.
The default pre-build value is 1.25 times the value of the build value.
.It Fl j Ar name
Run the bulk build on the jail named
.Ar name .
.It Fl k
Do not consider failures to be fatal
when using
.Fl t .
Do not skip dependent ports on findings.
This flag is automatically set when using
.Fl at .
.It Fl N
Do not build a package repository when the build is completed.
.It Fl NN
Do not commit the package repository when the build is completed.
This can be used to do a full test build but have the opportunity to delete
it all rather than publish it.
The packages will be stored in a
.Pa .building
directory that can be removed manually, otherwise the next build will
resume from that directory.
Depends on
.Cm ATOMIC_PACKAGE_REPOSITORY
being set to
.Sy yes .
.It Fl n
Dry run.
Show what would be done, but do not actually build or delete any
packages.
.It Fl O Ar overlay
Specify an extra ports tree to use as an overlay.
Multiple
.Fl O Ar overlay
arguments may be specified to stack them.
.Pp
These overlays should be setup with
.Xr poudriere-ports(8) .
.It Fl p Ar tree
Specify on which ports
.Ar tree
the bulk build will be done.
.Pq Default: Dq Li default
.It Fl R
Clean
.Va RESTRICTED
packages after building.
.It Fl r
Recursively test all dependencies as well.
This flag is automatically set when using
.Fl at .
.It Fl S
Do not recursively rebuild packages affected by other packages requiring
incremental rebuild.
This may result in broken packages if the ones they depend on are updated,
are not ABI-compatible, and were not properly
.Va PORTREVISION
bumped.
.It Fl T
Try building
.Va BROKEN
ports by defining
.Va TRYBROKEN
for the build.
.It Fl t
Add some testing to the specified ports.
Add
.Fl r
to recursively test all port dependencies as well.
When used with
.Fl a
then
.Fl rk
are implied.
.It Fl v
Enable additional information to be shown during the build.
Specify twice to enable debug output.
.It Fl w
Save
.Va WRKDIR
on build failure.
The
.Va WRKDIR
will be tarred up into
.No ${ Ns Va POUDRIERE_DATA Ns } Ns Pa /wrkdirs .
.It Fl z Ar set
This specifies which SET to use for the build.
See the
.Em CUSTOMIZATION
section in
.Xr poudriere 8
for examples of how this is used.
.El
.Sh ENVIRONMENT
.Bl -tag -width "POUDRIERE_INTERACTIVE_NO_INSTALL"
.It Ev POUDRIERE_INTERACTIVE_NO_INSTALL
If specified, the package is not installed in interactive mode.
.It Ev TERM
Passed through for interactive modes.
.El
.Sh EXAMPLES
.Bl -tag -width 0n
.It Sy Example 1\&: No Starting a Bulk Build
.Pp
The following example starts a bulk build of two ports.
.Bd -literal -offset 2n
.Li # Ic poudriere bulk accessibility/sct www/firefox
.Ed
.El
.Sh SEE ALSO
.Xr poudriere 8 ,
.Xr poudriere-distclean 8 ,
.Xr poudriere-image 8 ,
.Xr poudriere-jail 8 ,
.Xr poudriere-logclean 8 ,
.Xr poudriere-options 8 ,
.Xr poudriere-pkgclean 8 ,
.Xr poudriere-ports 8 ,
.Xr poudriere-queue 8 ,
.Xr poudriere-status 8 ,
.Xr poudriere-testport 8 ,
.Xr poudriere-version 8
.Sh AUTHORS
.An Baptiste Daroussin Aq bapt@FreeBSD.org
.An Bryan Drewery Aq bdrewery@FreeBSD.org