Creating Third-Party Library Configuration Files for Waf

Lumberyard’s Waf build system has the ability to incorporate third-party libraries. Waf’s uselib mechanism can apply the proper dependency injection of a third-party library into any project or module. In the wscript file, the uselib attribute specifies all-caps identifiers that represent the third-party libraries. The identifiers are defined in the .json configuration files that can exist either in the global _WAF_\3rdParty directory or in the gem-specific 3rdParty directory. These configuration files provide details for the library, including include paths, library paths, and linkage information. The files also specify whether the library is shared, dynamic, or header-only.

Topics

• Key Features
• Supported Library Types
• Using Aliases
• Creating Entries for Multiple Related Libraries
• Third-Party Library Configuration File Attributes
• Special Reserved Words

Key Features

Key features of the uselib mechanism include centralized management and an abstraction layer that makes it easy to use the same libraries across Lumberyard projects and modules.

Centralized Management

Encapsulating the build details within a single file helps centralize the management of third-party libraries. Centralized management in a single file is also beneficial when changes like different include paths or libraries to the third-party library occur. These configuration files can be global to the entire engine, or gem-specific based on the gems that are enabled for the game.

Abstraction

The third-party identifiers provide an abstraction layer between the third-party library details and the consuming project or module. This layer makes it unnecessary to explicitly inject the paths for the library and include headers for each project or module that uses the library. Instead, the uselib mechanism with the third-party identifier provides the necessary injection of paths, libraries, and custom rules.

For example, assume you have two third-party libraries, StaticLibA and DynamicLibB, that you want to link into test project ProjectOne. Without the uselib system, the wscript file for ProjectOne would look like the following:

Example
ProjectOne\wscript

def build(bld):
    bld.LumberyardSharedLibrary(
        target = 'ProjectOne',
        ...
        includes = ['3rdParty/StaticLibA/includes',
                    '3rdParty/DynamicLibB/includes'],
        win_stlibpath = ['3rdParty/StaticLibA/win32/lib'],
        darwin_stlibpath = ['3rdParty/StaticLibA/osx/lib'],
        ...
        win_libpath = ['3rdParty/DynamicLibB/win32/bin'],
        darwin_libpath = ['3rdParty/DynamicLibB/osx/bin'],
        ...
        lib = ['staticliba', 'dynamiclibb'],
        ...
        win_copy_dependent_files = ['3rdParty/DynamicLibB/win32/bin/dynamiclibb.dll'],
        darwin_copy_dependent_files = ['3rdParty/DynamicLibB/win32/bin/libdynamiclibb.dylib'],
        ...
    )

Instead, you could use the following third-party configuration files for the two libraries, as in the following examples:

Example
StaticLibA.json

{
    "name": "STATIC_LIB_A",
    "source": "3rdParty",
    "description": "My Static Library A",
    "includes": "includes",
    "platform": {
        "win_x64_vs2017": {
            "libpath": [
                "win32/lib"
            ],
            "lib": ["staticliba.lib"],
        },
        "darwin_x64": {
            "libpath": [
                "osx/lib"
            ],
            "lib": ["libstaticlibb.a"],
        }
    }
}

Example
DynamicLibB.json

{
    "name": "DYNAMIC_LIB_B",
    "source": "3rdParty",
    "description": "My Dynamic Library B",
    "includes": "includes",
    "platform": {
        "win_x64_vs2017": {
            "importlibpath": [
                "win32/lib"
            ],
            "import": [
                "dynamiclibb.lib"
            ],
            "shared": [
                "dynamiclibb.dll"
            ]
        },
        "darwin_x64": {
            "importlibpath": [
                "osx/lib"
            ],
            "import": [
                "libdynamiclibb.a"
            ],
            "shared": [
                "libdynamiclibb.dylib"
            ]
        }
    }
}

This now simplifies the wscript declaration for ProjectOne to the following:

Example
ProjectOne\wscript

def build(bld):
    bld.LumberyardSharedLibrary(
        target = 'ProjectOne',
        ...
        uselib = ['STATIC_LIB_A', 'DYNAMIC_LIB_B'],
        ...
    )

After you have created these third-party configuration files, you can apply the same simplification to any project or module in Lumberyard.

Supported Library Types

Third-party configuration files support the following library types:

• Header Only
• Static
• Shared/Dynamic

Note
For simplicity, the rest of this topic refers to shared/dynamic libraries as ‘shared’.

Header-only Libraries

Header-only libraries provide only the include paths to the dependent project or module. The following example third-party library configuration file contains a definition for a header-only library:

Example
rapidxml.json

{
    "name": "rapidxml",
    "source": "@3P:rapidxml@",
    "description": "Rapid XML header only library",
    "header_only": "True",
    "includes": [
        "include"
    ],
    "defines": [],
    "lib_required": "False"
}

This simple declaration specifies only the includes header path which is a subdirectory of the source directory. The source directory is defined by the alias pattern @3P:XXX@. For descriptions of the other attributes in this file, see Third-Party Library Configuration File Attributes.

Static Libraries

Static third-party library configuration files declare header include paths, library include paths, and library names for inclusion in a dependent project or module. The following example third-party library configuration file contains a declaration for a simple static library.

Example
lz4.json

{
    "name": "lz4",
    "source": "@3P:lz4@",
    "description": "LZ4 Compression Library",
    "includes": [
        "lib"
    ],
    "defines": [],
    "lib_required": "True",
    "shared_required": "False",
    "platform": {
        "win_x64_vs2017": {
            "libpath_debug": [
                "build/win_x64/vc140/debug"
            ],
            "libpath_release": [
                "build/win_x64/vc140/release"
            ],
            "lib": [
                "lz4.lib"
            ]
        },
        "win_x64_vs2019": "@win_x64_vs2017", 
        ...

Platform-Specific Information

In addition to the attributes described for header-only library definition, the preceding example includes platform-specific values. The platform attribute contains a dictionary of target platform sections. Each target platform must be specified here. If a particular target platform is not listed, this third-party library definition will not apply for that platform when built. Static libraries must specify the library path (libpath) and the names of the library file (lib) against which to link for each platform. The configuration extension for libpath attribute in the preceding example will be described later in this topic.

Shared Libraries

Shared libraries are more complex than static libraries. Shared libraries must declare an include path, import library path, shared library path, shared library file names, and optionally, program database (.pdb) files. The following example third-party library configuration file contains a definition for a shared library.

Example
boostpython.json

{
   "name": "boostpython",
   "source": "@3P:boost@",
   "description": "Boost header library including python support libraries",
   "defines": [],
   "lib_required": "False",
   "platform": {
      "win_x64_vs2017": {
         "includes": [
            "."
         ],
         "importlibpath": [
            "lib/windows/x64"
         ],
         "sharedlibpath": [
            "lib/windows/x64"
         ],
         "import_debug": [
            "boost_python-vc140-mt-gd-1_61.lib"
         ],
         "import_release": [
            "boost_python-vc140-mt-1_61.lib"
         ],
         "shared_debug": [
            "boost_python-vc140-mt-gd-1_61.dll"
         ],
         "shared_release": [
            "boost_python-vc140-mt-1_61.dll"
         ],
         "pdb_debug": [
            "boost_python-vc140-mt-gd-1_61.pdb"
         ]
      },
      ....

The following table describes the attributes for shared library configuration files.

Besides providing the paths for header inclusion and linking, Lumberyard Waf also copies the shared library and PDB files to the path where the dependent project or module is built.

Mixed Shared and Static Libraries

In some cases, the third-party library is static on some platforms but shared on others. For cases like these, it is possible to declare mixed static and shared libraries for each platform.

Using Aliases

The following sections describe how to simplify the configuration file by using aliases for libraries, platforms, and paths.

Aliasing Library Lists

The same list of libraries might be needed in different platform sections of the configuration file. To avoid repetition, you can define and use a list alias that expands to a list of other values. This is helpful for cases in which you have multiple shared libraries that import libraries of the exact same name.

You define an alias in the aliases node under the root node in the configuration file. Add the name of the alias to the aliases node and a list of strings that the alias represents under the alias name.

In the following example, the all_foo_lib_names alias represents the libraries foo_library_A through foo_library_D. The library names must not have any prefixes or extensions.

Note
For some platforms, the file name to library name convention is different. For example, Linux-based libraries prefix the file name with lib, but Windows-based libraries do not.

In the platform section, the shared library definitions for import, shared, and pdb use the notation ${} to form the entries, where is the name of the alias that was defined in the aliases section.

...
"aliases": {
    "all_foo_lib_names": [
        "foo_library_A",
        "foo_library_B",
        "foo_library_C",
        "foo_library_D"
    ]
},
"platform": {
	"win_x64_vs2017": {
		...
		"import": {
			"${all_foo_lib_names}.lib"
		},
		"shared": {
			"${all_foo_lib_names}.dll"
        },
        "pdb": {
			"${all_foo_lib_names}.pdb"
		}
        ...
     },
...

Aliasing Platform Declarations

Some different target platforms are binary compatible. In these cases, you do not have to declare a copy of an existing configuration section. Instead, you can use the platform attribute string alias @<platform_name> instead of another attribute dictionary.

In the following example, the win_x64_vs2017 and win_x64_vs2019 platforms are binary compatible, so the alias @win_x64_vs2017 is used.

"platform": {
    "win_x64_vs2017": {
        "libpath_debug": [
            "build/win_x64/vc141/debug"
        ],
        "libpath_release": [
            "build/win_x64/vc141/release"
        ],
        "lib": [
            "foo.lib"
        ]
    },
    "win_x64_vs2019": "@win_x64_vs2017",
...

Aliasing Paths

Path aliasing for the source attribute is common within the configuration file. The alias value is denoted by @XXXX@, where the XXXX value resolves to an absolute path. The third-party configuration file recognizes two different path alias types:

• @3P:YYYY@ – This alias refers to a third-party library that is managed by the Lumberyard Setup Assistant. These third-party libraries are globally available to the engine. Lumberyard Setup Assistant manages the location and version of these libraries independently from the build system. YYYY refers to the SDK identifier used by Lumberyard Setup Assistant and is different from the third-party uselib identifier used by Waf.
• @GEM@ – This alias is valid only for gem-defined third-party libraries and refers to the base path of the gem from which the third-party library is defined. For information about putting third-party libraries in gems, see Adding Third-Party Libraries.

Creating Entries for Multiple Related Libraries

Your project might have multiple libraries under a shared root source directory that are related but have separate purposes. This section shows how your configuration files can handle these more complex cases so that you can still take advantage of the uselib mechanism.

Build Configuration Filtering

Attributes can be constrained by build configuration. For example, a debug library and a release library might have the same name but exist in two different directories, as in the following example file structure:

/StaticLibA/
    /Include
        static_a.h
    /lib
        /debug
            static_a.lib
        /release
            static_a.lib

The following example shows the configuration file declaration for this file structure. Notice that the lib attribute is specified once, but the libpath attribute is specified twice.

"include": ["Include"],
"libpath_debug": ["lib/debug"],
"libpath_release": ["lib/release"],
"lib": ["static_a.lib"]

If the libraries are in the same directory, but named differently, the file structure might look like this:

/StaticLibA/
    /Include
        static_a.h
    /lib
        static_aD.lib
        static_a.lib

In this case, the value for libpath in the configuration file is the same, but there are two values for lib:

"include": ["Include"],
"libpath": ["lib"],
"lib_debug": ["static_aD.lib"],
"lib_release": ["static_a.lib"]

Specifying Multiple Identifiers

A third-party library might contain a suite of libraries and be organized by specific categories according to need. In these cases, the configuration file can define more than one identifier. As long as the library files are under the same directory that is specified for the source attribute, the configuration can specify multiple third-party identifiers.

The following example configuration file defines two third-party identifiers that uselib will point to: water_lib and air_lib. The lib attribute is prefixed with the name of the third-party identifier. Both libraries are under the same elements library directory and exist individually as water_lib.lib and air_lib.lib. Both water_lib and air_lib require the same base element_base.lib library.

{
	"name": [
		"water_lib",
		"air_lib"
	],
	"source": "elements",
	...
	"platform": {
		"win_x64_vs2017": {
			"libpath":	[
				"lib"
			],
			"water_lib/lib": [
				"element_base.lib",
				"water_lib.lib"
			],
			"air_lib/lib": [
				"element_base.lib",
				"air_lib.lib"
			]
			...
		}
	...
}

Third-Party Library Configuration File Attributes

The following attributes are used in third-party library definition files.

Special Reserved Words

The following special reserved words are used in third-party library configuration files.

…
“linkflags”: [
    “-force_load”,
    “%LIBPATH(libtomcrypt.a)"
]