After bringing down my full rebuild time from 30 seconds to 2 seconds in my compiler, I thought it useful to note down some learned lessons.

Low risk changes

Keep code out of headers

Avoid inline/static functions in headers.

Prefer forward declarations over includes

If you dont absolutely need the full definition of a type then prefer forwarding it. In C++ you dont need the definition of a type to pass it by reference.

No relative include paths

Either provide the compiler with the include paths properly or reoganize your headers to not need this. Often an antipattern anyway.

More involved changes

Seperate templates into headers and inline files

Often you'll only need the definition of a template in a header file, and not the full type. If you declare the template type as you would usually, but then implement the functions in a seperate file you can avoid alot of include overhead.

                
                    // vector.hpp
                    // old: all in one file
                    template <typename T> class Vector {
                        T *m_front;
                        T *m_back;
                        T *m_capacity;
                    public:
                        T& get(int index) { return *(m_front + index); }

                        // rest of the impl...
                    };
                
                
                    // vector.hpp
                    // new: only the declarations
                    template <typename T> class Vector {
                        T *m_front;
                        T *m_back;
                        T *m_capacity;
                    public:
                        T& get(int index);

                        // rest of the function declarations
                    };
                
                
                    // vector.inl
                    template<typename T>
                    T& Vector<T>::get(int index) { return *(m_front + index); }

This way you can include just the `.hpp` file in your headers, then the `.inl` in your source files where you actually need to instantiate those functions.

The stuff that wont make it past code review

Rely on transitive header includes

In wysiwyg style includes you'll often pull in the same headers across multiple files.

                
                    - system.h. uses vectors, strings, and core
                        - vector.h
                        - string.h
                        - core.h

                    - game.h. uses system, vectors, and core
                        - system.h
                        - vector.h
                        - core.h

Relying on transitive deps avoids the compiler needing to query to fs as often. pragma once isnt free, especially with relative includes.

                
                    - system.h. uses vectors, strings, and core
                        - vector.h
                        - string.h
                        - core.h

                    - game.h. uses system, vectors, and core
                        - system.h pulls in vector.h and core.h so we dont include them ourselves

Dont declare anything except mandatory dependencies as exports in your build system

Often times your modules of code will provide a few related apis, most other components will not commonly use a modules entire api surface. Constructing an object may require an allocator, but using it may only require a map. This means that a module that only ever constructs this type only needs to depend on arenas being built. And a module that only ever uses it only needs to depend on maps being built. Not declaring all possibly used modules as dependencies in your build system allows builds and links to happen sooner.

Final results

This graph represents 2 seconds wall clock time on a 5950x, building 180 source files totalling 12000 significant lines of C. The final build times

Tools to diagnose build bottlenecks

NinjaTracing for build generators that produce ninja files.
Perfetto for viewing the traces.
ClangBuildAnalyzer if you're using clang.

See the code

The compiler