diff options
author | Nate McMaster <natemcmaster@users.noreply.github.com> | 2017-12-02 05:15:17 +0300 |
---|---|---|
committer | GitHub <noreply@github.com> | 2017-12-02 05:15:17 +0300 |
commit | cf7eccab5eec7d68ba3f9c201c5a109bb70036dd (patch) | |
tree | dc65e7251a4c811badc6f76370967627e06e7323 | |
parent | d5ffe85bb906b05771228134e0f1fbdd32ea8414 (diff) |
Add a design document for the bill of materials and build caching (#688)
[ci skip]
-rw-r--r-- | docs/BillOfMaterials.md | 171 | ||||
-rw-r--r-- | docs/BuildCaching.md | 30 |
2 files changed, 201 insertions, 0 deletions
diff --git a/docs/BillOfMaterials.md b/docs/BillOfMaterials.md new file mode 100644 index 0000000000..d56a05995e --- /dev/null +++ b/docs/BillOfMaterials.md @@ -0,0 +1,171 @@ +Bill of Materials +================= + +The bill of materials (bom) is a list of artifacts produced by the build. + +## Generating a new bom + +A new bill of materials is generated on each build and placed in `artifacts/bom.$(Version).xml`. + +In addition, this bill of materials can be produced without building the repo by running `build.ps1 -bom`. + +## Configuration data + +:x: The bom SHOULD NOT contain data that is for configuration. Configuration data is any kind of parameter that a consumer needs to find the artifact. + +Examples of ephemeral data: + + - File paths + - Download links + - Passwords + +## Format + +The bill of materials is an XML file. Here is a minimal example of a bom. + +```xml +<Build> + + <Artifacts> + <Artifact Id="Microsoft.Extensions.Configuration.Json.2.0.0.nupkg" Type="NuGetPackage" /> + </Artifacts> + + <Artifacts Category="noship"> + <Artifact Id="Experimental.1.0.0-alpha.123.nupkg" Type="NuGetPackage" /> + </Artifacts> + + <Dependencies> + <Link Source="Microsoft.Extensions.Configuration.Json.2.0.0-rtm-123.nupkg" Target="Newtonsoft.Json.10.0.1.nupkg" /> + <Link Id="Experimental.1.0.0-alpha.123.nupkg" Target="Microsoft.Extensions.Configuration.Json.2.0.0-rtm-123.nupkg" /> + </Dependencies> + +</Build> +``` + +### Elements + +#### `<Build>` + +Describes the build output. The root element of the document. + +Optional attributes: + - `Id` - a unique identifier describing this build. + - `Created` - a timestamp of when the build was created. + +#### `<Artifacts>` + +Describes a group of artifacts + +Optional attributes: + - `Category` - an arbitrary category name. + +#### `<Artifact>` + +Describes a file produced by the build. + +Required attributes: + - `Id` - the file name of the artifact. Id MUST BE unique per each bill of materials. + - `Type` - an arbitrary string describing the semantic type of the artifact. + +Optional attributes: + - Others - any other attributes on the element are considered metadata values about the artifact. + +#### `<Dependencies>` + +Describes artifact dependencies. + +#### `<Link>` + +Describes a dependency between two artifacts. + +Requried attributes + - `Source` - the artifact that has a dependency. This artifact MUST BE defined by an `<Artifact>` in the same `<Build>` + - `Target` - the thing it depends on. This artifact MAY OR MAY NOT be described in this `<Build>`. + It might have been produced in a previous build or be produced by a partner team. + +## Artifact types + +These are some common artifact types produced by the build. + + - NuGetPackage + - NuGetSymbolsPackage + - DebianPackage + - RedhatPackage + - ZipArchive + - TarGzArchive + - TextFile + - LzmaArchive + - TestBundle + +## Artifact categories + +Tools may use the category information to define how they handle different files. + +These are common categories for artifacts used in aspnet/Universe. + + - ship - used for files that will be published to NuGet.org, microsoft.com, or other official publication locations + - shipoob - used for files that will be distributed in other mechanism. + - noship - these files should not be published to partner teams or released publically. This may include test artifacts. + +## A more complete example + +```xml +<Build Id="(commithash)+Release+Signed+123" Created="12/1/2017 5:10:06 PM +00:00"> + + <Artifacts Category="ship"> + <Artifact Id="Microsoft.Extensions.DependencyInjection.2.0.0-rtm-123.nupkg" Type="NuGetPackage" + ShouldBeSigned="true" + PackageId="Microsoft.Extensions.DependencyInjection" + Version="2.0.0-rtm-123" + BranchName="release/2.0.0" + CommitHash="xyz123" + RepositoryUrl="https://github.com/aspnet/DependencyInjection" + FileHash="ABCDEF0123456789" /> + <Artifact Id="Microsoft.AspNetCore.2.0.0-rtm-123.nupkg" Type="NuGetPackage" + ShouldBeSigned="true" + PackageId="Microsoft.AspNetCore" + Version="2.0.0-rtm-123" /> + <Artifact Id="Microsoft.AspNetCore.2.0.0-rtm-123.symbols.nupkg" Type="NuGetSymbolsPackage" + ShouldBeSigned="true" + PackageId="Microsoft.AspNetCore" + Version="2.0.0-rtm-123" /> + <Artifact Id="aspnetcore-store-2.0.0-rtm-123-win-x64.zip" Type="ZipArchive" + ShouldBeSigned="false" + RuntimeIdentifier="win-x64" + Version="2.0.0-rtm-123" /> + <Artifact Id="aspnetcore-store-2.0.0-rtm-123-linux-x64.deb" Type="DebianPackage" + ShouldBeSigned="false" + RuntimeIdentifier="linux-x64" + Version="2.0.0-rtm-123" /> + </Artifacts> + + <Artifacts Category="noship"> + <Artifact Id="Experimental.1.0.0-alpha.123.nupkg" Type="NuGetPackage" + PackageId="Experimental" + Version="1.0.0-alpha.123+commithash:123456" /> + <Artifact Id="Microsoft.AspNetCore.Tests.zip" Type="TestBundle" + RuntimeFramework="netcoreapp2.0" + RuntimeFrameworkVersion="2.0.3" /> + </Artifacts> + + <Dependencies> + <Link Source="Microsoft.Extensions.DependencyInjection.2.0.0-rtm-123.nupkg" Target="Microsoft.Extensions.Common.2.0.0.nupkg" /> + <Link Source="Microsoft.AspNetCore.2.0.0-rtm-123.nupkg" Target="Newtonsoft.Json.9.0.1.nupkg" /> + <Link Source="Microsoft.AspNetCore.2.0.0-rtm-123.nupkg" Target="Microsoft.Extensions.DependencyInjection.2.0.0-rtm-123.nupkg" /> + <Link Source="aspnetcore-store-2.0.0-rtm-123-linux-x64.deb" Target="Microsoft.Extensions.DependencyInjection.2.0.0-rtm-123.nupkg" /> + <Link Source="aspnetcore-store-2.0.0-rtm-123-linux-x64.deb" Target="Microsoft.AspNetCore.2.0.0-rtm-123.nupkg" /> + </Dependencies> + +</Build> +``` + +### Example usage: signing + +In the example above, some of the `<Artifact>` items were marked `ShouldBeSigned="true"`. Our signing tool could use this as a way to +determine which files should be passed on to signing. + +### Example usage: metadata + +In the example above, some of the artifacts could contain data like FileHash, CommitHash, RepositoryUrl, BranchName, and others. +It is up to the consumer of the bom to define how to interpret and use this. + diff --git a/docs/BuildCaching.md b/docs/BuildCaching.md new file mode 100644 index 0000000000..6c746807b8 --- /dev/null +++ b/docs/BuildCaching.md @@ -0,0 +1,30 @@ +Build Caching +============= + +This repository can build the entire ASP.NET Core stack from scratch. However, in most cases, we only want to build part of the stack. + +## Building with cached artifacts + +This is the default behavior when `build.ps1` without arguments. + +This repository may contains files under `releases/*.xml`. These files follow the [Bill of Materials](./BillOfMaterials.md) format. +Any artifacts described in these bom's are assumed to have been built in a previous release and are available for download. +The build may use this information to skip building certain assets when a build action would produce a duplicate asset. + +## Disabling caching + +You can for the build to skip reading the bill of materials and build everything from scratch by calling `build.ps1 -NoCache`. +This will cause the build to always rebuild assets. + +## When to commit a new bill of material + +Each build will produce a new Bill of Materials (bom). Normally, this bom isn't added to source code right away. +This bom should only be committed to source once the assets it describes are published and available from caching mechanisms. + +## Caching mechanisms + +These are some common caching mechanisms used to store build artifacts. + + - NuGet servers. NuGet.org, MyGet.org, and file shares can be used to restore previously-built nupkg files + - Debian and RPM feeds. Some .deb and .rpm installers may be cached on https://packages.microsoft.com/repos. + - Azure blob feeds. Arbitrary build artifacts may be cached on https://aspnetcore.blob.core.windows.net/. |