Repository navigation
Expand file tree
/
Copy pathspirv.cppm
More file actions
934 lines (871 loc) · 46.2 KB
/
Copy pathspirv.cppm
File metadata and controls
934 lines (871 loc) · 46.2 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
// mcpp.rules.spirv — how a GLSL translation unit becomes SPIR-V, stated once.
//
// THE MODULE NAME IS `mcpp.rules.<x>` AND THE PACKAGE NAMESPACE IS `mcpp`.
// Both halves are the rule-package specification, not a preference. The module
// name is declared by this source rather than derived from the package name
// (spec I1), and `mcpp.*` is reserved for rules the mcpp project maintains
// (I8) -- which is enforced as a warning keyed on the package NAMESPACE, so a
// rule carrying this module name under any other namespace is told that it
// claims an origin it does not have.
//
// The division of labour is the one `mcpp.rules.cuda` established, and it is the
// point of both packages: the ENGINE owns the graph — the accelerator axis,
// the constrained source globs that route `shaders/*.comp` here instead of to
// the C++ compiler, the action edges and their ordering, the fingerprint — and
// does not know the word "vulkan" or the word "glslang". The RULE owns the
// spelling: which compiler, which flags, what the generated symbol is called.
//
// WHAT COMES OUT IS A HEADER, NOT AN OBJECT.
//
// A SPIR-V module is data the program hands to `vkCreateShaderModule`, not
// code the linker places. Two shapes are possible: a `.spv` file beside the
// binary, which makes the program's correctness depend on its working
// directory, or a C array compiled into it. This rule emits the second, so the
// artifact carries its shaders and a `mcpp pack` of it has nothing further to
// collect.
//
// `role = "source"` is what makes that work. It is the one role the engine
// orders BEFORE compilation (`action_precedes_compilation`), which is exactly
// what a generated header needs and exactly what an `artifact` role would not
// give: an artifact output is ordered against the LINK, and the header has to
// exist before the first translation unit that includes it is compiled.
//
// TWO COMPILERS, AND THE RULE STATES WHICH ONE IT USED.
//
// glslang and glslc are the two reference GLSL compilers and their command
// lines are not interchangeable. glslang's `-x --vn <name>` emits a COMPLETE C
// declaration -- `const uint32_t <name>[] = { ... };` -- while glslc's
// `-mfmt=c` emits a BARE INITIALISER LIST, `{ ... }`, which is not a
// translation unit on its own. So on both routes the compiler's output is a
// `<base>.inc` and the PUBLIC header `<base>.h` is written by this rule --
// `write_header` below. It has to be that way round for a second reason: a
// generated header must be includable on its own, and glslang's complete
// declaration names `uint32_t` while including nothing.
//
// An earlier revision of this file supported glslang alone, and said why:
// nothing in this ecosystem published glslc, "and a route with no payload
// behind it is a claim rather than a feature". `xim:shaderc` now publishes it,
// so the route exists. Neither is a default over the other. Discovery takes
// what the project named, then what the environment named, then whichever
// payload the workspace installed, and the build log states which it found --
// two compilers that produce an equivalent header from one shader are still
// two different answers to "what compiled this".
module;
#include <cctype>
#include <cstdio>
export module mcpp.rules.spirv;
import std;
import mcpp;
// The lib root, which carries `mcpp::plugins::surface` -- the declarations a
// consumer names, written once for every member that embeds a payload.
import mcpp.plugins;
import mcpp.plugins.declare;
import mcpp.plugins.tool;
// WHY NOTHING HERE USES `std::println`, AND WHY THAT IS NOT A STYLE CHOICE.
//
// `std::print` and `std::println` are not header-only. Both of their overloads
// reach into the libc++ DYLIB -- `__is_posix_terminal(FILE*)` for the stdout
// form and `__get_ostream_file(ostream&)` for the stream form -- and those
// symbols were added to that library in a version macOS 14 does not ship. A
// build program's link resolves `-lc++` to the system copy there, so a rule
// that printed with `std::println` compiled and then failed to link:
//
// ld64.lld: error: undefined symbol: std::__1::__is_posix_terminal(__sFILE*)
//
// naming neither the call that needed it nor the reason. Measured on
// macos-14; macos-15 has the symbol, which is why nothing saw this until a
// rule was first compiled on the older of the two supported releases.
//
// `std::format` is header-only and has no such dependency, so every message in
// this file is formatted and then streamed.
export namespace mcpp::rules::spirv {
struct options {
// The Vulkan environment the SPIR-V targets. Left empty, it is taken from
// the accelerator axis — `accel = "vulkan1.2"` in the manifest — which is
// the same route `mcpp.rules.cuda` takes for `sm_89`, and the reason neither
// rule needs a second place to state what the build is for.
std::string target_env;
// `-I` for GLSL `#include`, `-D` for its preprocessor. Relative entries
// resolve against the package root; an absolute entry is passed through.
std::vector<std::string> includes;
std::vector<std::string> defines;
// glslang's optimiser (`-Os`), which is spirv-opt linked into it.
bool optimize = true;
// The shader compiler (0.19.0: a `tool::choice`, so a path still
// assigns): glslang or glslc, told apart by program name. Default: the
// payload this rule declares for the host (`xim:glslang` on Linux,
// `xim:shaderc` on macOS and Windows), or an override of it.
mcpp::plugins::tool::choice compiler;
std::string out_dir = std::string(mcpp::out_dir());
// ── What a consumer names ────────────────────────────────────────────────
//
// The declarations are written by `mcpp.plugins.surface`, which states why
// they take the shape they do. What this rule decides is only which surface
// and under which name.
// `module_` or `c_header`. The default follows `[language] modules`, which
// mcpp reports in `MCPP_LANGUAGE_MODULES`; an engine that does not report it
// leaves the header surface in place, so an older engine keeps the behaviour
// every consumer of this package had before the surface existed.
mcpp::plugins::surface::kind surface = mcpp::plugins::surface::default_surface();
// Where the compiled SPIR-V lives. `header` compiles it in as generated
// source, `object` as a section reached through `.incbin`, `sidecar` as a
// file beside the artifact. See `mcpp::plugins::surface::storage` for the
// measurement that makes `header` the default.
//
// It changes what this rule asks the compiler for. Under `header` the
// compiler is told to emit a C declaration (`-mfmt=c`, `-x --vn`); under
// the other two it emits a bare `.spv`, which is both simpler and the one
// shape both compilers agree on.
mcpp::plugins::surface::storage storage = mcpp::plugins::surface::storage::header;
// The module a consumer imports, and the namespace the declarations sit in:
// `myapp.shaders` gives `myapp::shaders::blur_comp()`. Empty derives it from
// the PACKAGE NAME -- `[package] name`, not the directory the package
// happens to sit in -- so a project that states nothing still gets a name
// no other package in the build can claim. The two are different questions
// whenever a package is laid out under a generic directory, and
// `mcpp::plugins::surface::module_root_for` records what it cost to answer
// the wrong one.
std::string module_name;
// The directory shader paths are made relative to when deriving namespaces.
// `shaders/post/tonemap.frag` under a base of `shaders` becomes
// `myapp::shaders::post::tonemap_frag`. Empty derives the base from the
// shallowest directory every shader shares, which is what a project that
// globs one tree already means and is why this is rarely written.
std::string base_dir;
};
// Where the generated headers are written, and what the build program passes
// to `mcpp::include_dir` so `#include "scale_comp.h"` resolves.
inline std::string include_dir(const options& opt) {
return (std::filesystem::path(opt.out_dir) / "spirv").string();
}
// ─── What the engine said ──────────────────────────────────────────────────
// The `vulkan` chunk of `mcpp::accel()`. Unlike CUDA's, it carries no
// architecture set: SPIR-V is the portable form, and which GPU executes it is
// decided when the driver compiles it, not here. A rule that demanded an
// architecture would be inventing a requirement its device API does not have.
struct target {
std::string version; // "1.2", from `vulkan1.2`
bool present = false;
};
inline std::string_view trim(std::string_view s) {
while (!s.empty() && (s.front() == ' ' || s.front() == '\t')) s.remove_prefix(1);
while (!s.empty() && (s.back() == ' ' || s.back() == '\t')) s.remove_suffix(1);
return s;
}
// Split on one character. Written out rather than taken from <ranges> for the
// reason `mcpp.rules.cuda` records: GCC 16 refuses the ranges split view
// instantiated inside an exported inline function when build.mcpp imports the
// module, and clang does not.
inline std::vector<std::string_view> split(std::string_view s, char sep) {
std::vector<std::string_view> out;
for (std::size_t i = 0; i <= s.size();) {
auto j = s.find(sep, i);
out.push_back(s.substr(i, j == std::string_view::npos ? s.size() - i : j - i));
if (j == std::string_view::npos) break;
i = j + 1;
}
return out;
}
inline target parse_target(std::string_view accel) {
target t;
for (std::size_t i = 0; i <= accel.size();) {
auto comma = accel.find(',', i);
auto chunk = trim(comma == std::string_view::npos ? accel.substr(i)
: accel.substr(i, comma - i));
i = comma == std::string_view::npos ? accel.size() + 1 : comma + 1;
if (!chunk.starts_with("vulkan")) continue;
t.present = true;
auto rest = chunk.substr(std::string_view("vulkan").size());
// A bare `vulkan` is legitimate and means "whatever the loader offers";
// the target environment then falls back to the default below.
auto plus = rest.find('+');
t.version = std::string(trim(plus == std::string_view::npos ? rest
: rest.substr(0, plus)));
}
return t;
}
// ─── The compiler ──────────────────────────────────────────────────────────
inline bool is_file(const std::string& p) {
std::error_code ec;
return !p.empty() && std::filesystem::is_regular_file(p, ec);
}
// Discovery, in the order a project can predict: what it named, what the
// environment named, the payload the workspace installed, then the PATH. The
// PATH comes last on purpose — a host glslang is a fine fallback and a poor
// default, because it makes the SPIR-V depend on a machine rather than on a
// declaration.
// WHICH COMPILER, AND WHICH OF THE TWO IT IS.
//
// The flavour is decided by the program's own name rather than by asking it,
// because the answer is needed before any flag can be chosen and the two
// disagree about almost every flag. A path a project or the environment names
// is classified the same way, so `MCPP_GLSLC=/opt/bin/glslc` needs no second
// variable to say what it is.
enum class flavour { none, glslang, glslc };
struct compiler {
std::string path;
flavour kind = flavour::none;
// Set when discovery already said why it failed, so the caller does not
// follow a precise message with a generic one that contradicts it.
bool reported = false;
// The payload was asked for (on request): this program runs again with it.
bool pending = false;
explicit operator bool() const { return kind != flavour::none && !path.empty(); }
const char* name() const { return kind == flavour::glslc ? "glslc" : "glslang"; }
};
inline flavour classify(const std::string& path) {
const auto stem = std::filesystem::path(path).stem().string();
if (stem == "glslc") return flavour::glslc;
if (stem == "glslang" || stem == "glslangValidator") return flavour::glslang;
return flavour::none;
}
// HOW THIS HOST SPELLS A PROGRAM, decided where the build program is compiled.
//
// The build program runs on the machine doing the building, so these are
// properties of the HOST and not of the target being compiled for -- a cross
// build from Linux to Windows still looks for `glslc`, because that is the
// binary about to be executed.
#if defined(_WIN32)
inline constexpr std::string_view kExeSuffix = ".exe";
inline constexpr char kPathSep = ';';
#else
inline constexpr std::string_view kExeSuffix = "";
inline constexpr char kPathSep = ':';
#endif
// The first of `<dir>/<name>` and `<dir>/<name>.exe` that exists. Both are
// tried on every host rather than only the one whose suffix matches: a payload
// repacked with the other convention is then found instead of silently missed,
// and the cost is one `stat`.
inline std::string program_in(const std::filesystem::path& dir, std::string_view name) {
std::string bare(name);
for (auto const& n : { bare, bare + std::string(kExeSuffix) }) {
auto p = (dir / n).string();
if (is_file(p)) return p;
}
return {};
}
inline std::string first_on_path(const char* exe) {
const char* path = std::getenv("PATH");
if (!path || !*path) return {};
std::string_view sv(path);
for (std::size_t i = 0; i <= sv.size();) {
auto sep = sv.find(kPathSep, i);
auto dir = sv.substr(i, sep == std::string_view::npos ? sv.size() - i : sep - i);
i = sep == std::string_view::npos ? sv.size() + 1 : sep + 1;
if (dir.empty()) continue;
if (auto p = program_in(std::filesystem::path(dir), exe); !p.empty()) return p;
}
return {};
}
// DISCOVERY THROUGH `mcpp.plugins.tool` (0.19.0), in the order every member
// uses: what the build program named, the environment variables this rule has
// always read (`MCPP_GLSLC`, then `MCPP_GLSLANG`), the engine's override or
// payload (glslang, then shaderc). A name is classified by program name, so
// `MCPP_GLSLC=/opt/bin/glslc` needs no second variable to say what it is.
//
// THE PATH IS NO LONGER CONSULTED BY ITSELF. Until 0.18 it was the last
// resort, which made the SPIR-V depend on whatever the machine had; it is now
// a choice (`tool::on_path()`). The old fallback still answers, with a warning
// naming that choice, until 2027-04-01.
inline const mcpp::plugins::tool::spec& glslang_spec() {
static const mcpp::plugins::tool::spec s{
.who = "mcpp.rules.spirv", .package = "glslang",
.programs = {"glslangValidator", "glslang"}, .option = "options::compiler",
.legacy_env = "MCPP_GLSLANG" };
return s;
}
inline const mcpp::plugins::tool::spec& shaderc_spec() {
static const mcpp::plugins::tool::spec s{
.who = "mcpp.rules.spirv", .package = "shaderc", .programs = {"glslc"},
.option = "options::compiler", .legacy_env = "MCPP_GLSLC" };
return s;
}
inline compiler from_found(const mcpp::plugins::tool::found& f) {
if (f.pending()) return { .pending = true };
if (!f) return {};
auto k = classify(f.program);
if (k == flavour::none) {
std::cerr << std::format("mcpp.rules.spirv: '{}' is neither glslang nor glslc by program\n"
" name, and the two share almost no flags. Rename the program or point at the\n"
" real one.", f.program) << '\n';
return { .reported = true };
}
return { f.program, k };
}
inline compiler find_compiler(const options& opt) {
namespace tool = mcpp::plugins::tool;
if (!opt.compiler.is_default()) {
// A program is resolved under the spec its name selects; a root or
// PATH is tried for glslang's names, then glslc's.
if (opt.compiler.how == tool::choice::kind::program) {
const auto k = classify(opt.compiler.value);
if (k == flavour::none) {
std::cerr << std::format("mcpp.rules.spirv: options::compiler names '{}', which is neither glslang\n"
" nor glslc by program name, and the two share almost no flags. Rename the\n"
" program or point at the real one.", opt.compiler.value) << '\n';
return { .reported = true };
}
return from_found(tool::resolve(k == flavour::glslc ? shaderc_spec() : glslang_spec(),
opt.compiler));
}
if (auto c = from_found(tool::resolve(glslang_spec(), opt.compiler)); c) return c;
return from_found(tool::resolve(shaderc_spec(), opt.compiler));
}
for (auto* s : {&shaderc_spec(), &glslang_spec()})
if (const char* e = std::getenv(s->legacy_env.c_str()); e && *e)
return from_found(tool::resolve(*s));
for (auto* s : {&glslang_spec(), &shaderc_spec()}) {
auto quiet = *s;
quiet.legacy_env.clear();
auto f = tool::resolve(quiet);
if (f || f.pending()) return from_found(f);
}
for (const char* exe : {"glslangValidator", "glslang", "glslc"})
if (auto p = first_on_path(exe); !p.empty()) {
mcpp::warning(std::format(
"mcpp.rules.spirv: using '{}' from PATH, because nothing else names a shader "
"compiler. A compiler found on PATH is a choice the build program states: "
"`options::compiler = mcpp::plugins::tool::on_path();`. This fallback is "
"removed on 2027-04-01.", p).c_str());
mcpp::decision("tool:mcpp.rules.spirv:glslang", "path", p.c_str());
return { p, classify(p) };
}
return {};
}
// `Glslang Version: 11:15.1.0` — the first field is the SPIR-V generator
// magic, the second is the release. The release is what a floor compares, and
// stating it as a fact is what makes a build log answer "which compiler
// produced this SPIR-V" without anyone having to reproduce the build.
// `popen` is POSIX and Windows spells it `_popen`; the null device differs
// too. Both are named here so the call sites below read the same on every
// host -- the alternative is a `#if` around each one, and the one that gets
// forgotten is the one nobody compiles.
inline FILE* open_pipe(const std::string& cmd) {
#if defined(_WIN32)
return ::_popen(cmd.c_str(), "r");
#else
return ::popen(cmd.c_str(), "r");
#endif
}
inline void close_pipe(FILE* p) {
#if defined(_WIN32)
::_pclose(p);
#else
::pclose(p);
#endif
}
inline constexpr const char* kNullDevice =
#if defined(_WIN32)
"NUL";
#else
"/dev/null";
#endif
inline std::string run_and_capture(const std::string& cmd) {
FILE* p = open_pipe(cmd);
if (!p) return {};
std::string text;
char buf[512];
while (std::fgets(buf, sizeof buf, p)) text += buf;
close_pipe(p);
return text;
}
// THE OPTIMISER IS OPTIONAL AND ITS ABSENCE IS NOT A BUILD ERROR.
//
// glslang links spirv-opt only when built with `ENABLE_OPT`, and the payload
// this ecosystem publishes today is not:
//
// glslangValidator: Error: -Os not available; optimizer not linked
//
// Passing `-Os` to such a binary fails the compile. A shader that is
// unoptimised is still a correct shader, so the flag is dropped and the build
// says so once — refusing would make an optional pass a requirement, and
// passing it silently would let the manifest claim an optimisation that did
// not happen.
inline bool has_optimizer(const std::string& exe) {
const auto out = run_and_capture("\"" + exe + "\" -Os --version 2>&1");
return out.find("optimizer not linked") == std::string::npos;
}
inline std::string compiler_version(const compiler& cc) {
const std::string text = run_and_capture("\"" + cc.path + "\" --version 2>"
+ std::string(kNullDevice));
// glslc: `shaderc v2026.3 2fbab05...` on the first line. glslang:
// `Glslang Version: 11:15.1.0`, whose first field is the SPIR-V generator
// magic and whose second is the release.
const std::string_view key = cc.kind == flavour::glslc ? "shaderc v" : "Glslang Version:";
for (auto line : split(text, '\n')) {
auto at = line.find(key);
if (at == std::string_view::npos) continue;
auto rest = trim(line.substr(at + key.size()));
if (cc.kind == flavour::glslc) {
auto sp = rest.find(' ');
return std::string(sp == std::string_view::npos ? rest : rest.substr(0, sp));
}
auto colon = rest.find(':');
if (colon != std::string_view::npos) rest = rest.substr(colon + 1);
while (!rest.empty() && (rest.back() == '\n' || rest.back() == '\r'))
rest.remove_suffix(1);
return std::string(trim(rest));
}
return {};
}
// ─── Shaders ───────────────────────────────────────────────────────────────
// glslang infers the stage from the extension, and so does this table — but
// the table is consulted rather than trusted: an extension that is not a stage
// is refused by name instead of being handed to a compiler that will refuse it
// with a less specific message.
inline std::string_view stage_of(std::string_view ext) {
if (ext == ".comp") return "comp";
if (ext == ".vert") return "vert";
if (ext == ".frag") return "frag";
if (ext == ".geom") return "geom";
if (ext == ".tesc") return "tesc";
if (ext == ".tese") return "tese";
if (ext == ".mesh") return "mesh";
if (ext == ".task") return "task";
if (ext == ".rgen") return "rgen";
if (ext == ".rint") return "rint";
if (ext == ".rahit") return "rahit";
if (ext == ".rchit") return "rchit";
if (ext == ".rmiss") return "rmiss";
if (ext == ".rcall") return "rcall";
return {};
}
// `shaders/scale.comp` -> `scale_comp_spv`, and the header that declares it is
// `scale_comp.h`. Derived rather than configurable: a name a project chooses
// per shader is a name the project has to keep in agreement with its own
// `#include`, and this rule already decides the file name.
inline std::string symbol_of(std::span<const std::string> name_space,
std::string_view stem, std::string_view stage) {
std::string s;
for (auto const& seg : name_space) { s += seg; s += '_'; }
for (char c : stem)
s += (std::isalnum(static_cast<unsigned char>(c)) || c == '_') ? c : '_';
s += '_';
s += stage;
s += "_spv";
return s;
}
// The flat case, which is every shader that sits directly in the globbed tree.
inline std::string symbol_of(std::string_view stem, std::string_view stage) {
return symbol_of(std::span<const std::string>{}, stem, stage);
}
// The path-to-namespace derivations come from the lib root. They were written
// here and again in the other rule that has a namespaced surface; one function
// is what makes a directory named `default` get one answer rather than two that
// agree by inspection.
using mcpp::plugins::names::common_base_dir;
using mcpp::plugins::names::namespace_of;
// NEWLINE-SEPARATED, not `;`. A path may contain a semicolon and cannot
// contain a newline, which is why the engine chose it — and why a splitter
// that guesses wrong still works for exactly one shader and silently produces
// one impossible path for two.
// ─── This rule's share of the device sources ───────────────────────────────
//
// `mcpp::device_sources()` is the package's WHOLE device set, not this rule's
// share of it. A project with two backends puts a `.comp` and a `.cu` in one
// list, and every rule in that build program reads the same variable. Taking
// all of it works for exactly as long as a build has one rule in it, and then
// fails on the second -- not by dropping anything, but by handing a compiler a
// file it does not accept, with a message about that file's contents rather
// than about the rule that should have had it.
//
// So each rule takes the extensions it CLAIMS and leaves the rest to whoever
// claims those. A file no rule claims is not silently dropped either: the
// engine refuses a device source that reached no action, which is the one
// place that can see every rule's share at once.
constexpr std::string_view kClaimed[] = { ".comp", ".vert", ".frag", ".geom", ".tesc", ".tese", ".mesh", ".task",
".rgen", ".rint", ".rahit", ".rchit", ".rmiss", ".rcall",
// Stage-less, and claimed on purpose: `stage_of` refuses them by name
// and says which extensions carry a stage. Left unclaimed they would
// reach the engine's "no action compiles it" instead, which is true
// but says nothing about stages.
".glsl", ".hlsl", };
inline bool claims_extension(std::string_view path) {
const auto slash = path.find_last_of("/\\");
const auto name = slash == std::string_view::npos ? path : path.substr(slash + 1);
const auto dot = name.rfind('.');
if (dot == std::string_view::npos) return false;
const auto ext = name.substr(dot);
for (auto e : kClaimed) if (e == ext) return true;
return false;
}
inline std::vector<std::string> device_shaders() {
std::vector<std::string> out;
std::string_view all(mcpp::device_sources());
for (std::size_t i = 0; i <= all.size();) {
auto sep = all.find('\n', i);
auto one = trim(all.substr(i, sep == std::string_view::npos ? all.size() - i : sep - i));
i = sep == std::string_view::npos ? all.size() + 1 : sep + 1;
if (!one.empty() && claims_extension(one)) out.emplace_back(one);
}
return out;
}
// ─── The rule ──────────────────────────────────────────────────────────────
// glslc's `-mfmt=c` writes a BARE INITIALISER LIST and nothing else, so the
// declaration around it has to come from somewhere. It is written here rather
// than by a shell fragment in the action because an action is an argv, not a
// command line, and there is no shell in it to redirect or concatenate with.
//
// The header this writes is CONSTANT for a given shader name -- it names the
// symbol and includes the sibling the action produces -- so it is written at
// plan time, before any action runs. Its content does not depend on the
// shader's text, which is why nothing has to re-derive it when the shader
// changes: ninja rebuilds the `.inc`, the `#include` picks it up.
// THE PUBLIC HEADER IS WRITTEN BY THE RULE ON BOTH ROUTES, AND IT HAS TO BE
// INCLUDABLE ON ITS OWN.
//
// glslc emits an initialiser list, so a declaration had to be written around it
// and that header was self-contained by construction. glslang's `-x --vn` emits
// a complete C declaration, so the rule wrote nothing -- and that file names
// `uint32_t` while including nothing:
//
// tri_vert.h:3:7: error: 'uint32_t' does not name a type
//
// Measured in a sandbox, on a program whose first include was the generated
// header. Every consumer that had worked put a Vulkan header ahead of it, which
// is why an incomplete header read as a working one for as long as nobody
// included it first. Two compilers producing an EQUIVALENT header is the whole
// premise of this rule choosing between them, and "equivalent" has to include
// this.
//
// So both routes now produce the same two files: `<base>.inc` from the
// compiler, and `<base>.h` from here.
inline bool write_header(const std::string& header, const std::string& inc,
const std::string& sym, flavour kind) {
std::ofstream out{header, std::ios::trunc};
if (!out) {
std::cerr << std::format("mcpp.rules.spirv: cannot write {}", header) << '\n';
return false;
}
const auto incName = std::filesystem::path(inc).filename().string();
out << "// Generated by mcpp.rules.spirv.\n"
"#pragma once\n"
"#include <cstdint>\n";
if (kind == flavour::glslc) {
out << "// glslc emits an initialiser list; this declaration is what makes it\n"
"// a translation unit.\n"
"static const uint32_t " << sym << "[] =\n"
"#include \"" << incName << "\"\n"
";\n";
} else {
out << "// glslang emits a complete `const uint32_t " << sym << "[]`; what this\n"
"// adds is the type it names and a guard.\n"
"#include \"" << incName << "\"\n";
}
return out.good();
}
inline bool compile(std::span<const std::string> shaders, options opt = {}) {
if (shaders.empty()) return true;
const auto cc = find_compiler(opt);
// Asked for: the engine installs the payload and runs this program again.
if (cc.pending) return true;
if (!cc) {
if (cc.reported) return false;
std::cerr << std::format("mcpp.rules.spirv: no shader compiler found.\n"
" This rule DECLARES glslang, so a project normally writes nothing. Check, in "
"order:\n"
" mcpp older than 2026.9.6.6; `features = [\"rules-spirv\"]` missing from the\n"
" [build-dependencies] edge; or a build that names no Vulkan accelerator.\n"
" To use glslc instead, or to pin a different version, name it in your own\n"
" project and it wins:\n"
" [target.'cfg(accelerator = \"vulkan\")'.xlings.workspace]\n"
" \"xim:glslang\" = \"15.1.0\" # glslangValidator\n"
" \"xim:shaderc\" = \"2026.3\" # glslc\n"
"or name it: MCPP_GLSLANG=/path/to/glslangValidator, MCPP_GLSLC=/path/to/glslc,\n"
"or set options::compiler.") << '\n';
return false;
}
// The fact is keyed on the flavour, not on a shared name: which of the two
// compiled a shader is part of the answer, and a build log that recorded
// both under one key could not tell them apart.
if (const auto v = compiler_version(cc); !v.empty()) mcpp::fact(cc.name(), v.c_str());
bool optimize = opt.optimize;
// glslc always links its optimiser; glslang links spirv-opt only when
// built with ENABLE_OPT, and the payload this ecosystem publishes is not.
if (optimize && cc.kind == flavour::glslang && !has_optimizer(cc.path)) {
mcpp::warning("mcpp.rules.spirv: this glslang was built without spirv-opt "
"(-Os not available; optimizer not linked); shaders are "
"compiled unoptimised");
optimize = false;
}
auto env = opt.target_env;
if (env.empty()) {
const auto t = parse_target(mcpp::accel());
env = t.version.empty() ? "vulkan1.0" : "vulkan" + t.version;
}
const std::string root = mcpp::manifest_dir();
const auto gen = include_dir(opt);
std::error_code ec;
std::filesystem::create_directories(gen, ec);
// Where namespaces start counting from. Derived unless the project said,
// for the reason `common_base_dir` records.
const std::string baseDir = opt.base_dir.empty() ? common_base_dir(shaders)
: opt.base_dir;
// The module a consumer imports. `<package>.shaders` unless the project
// named one, so two packages in one build cannot claim the same module.
const std::string moduleName =
opt.module_name.empty()
? mcpp::plugins::surface::module_root_for(
mcpp::package_name(),
std::filesystem::path(mcpp::manifest_dir()).filename().string())
+ ".shaders"
: opt.module_name;
// TWO SHADERS THAT DIFFER ONLY BY DIRECTORY PRODUCE ONE HEADER AND ONE
// SYMBOL, AND THAT HAS TO BE REFUSED HERE.
//
// The output name is the stem and the stage, as this rule documents, so
// `shaders/ui/text.vert` and `shaders/world/text.vert` both resolve to
// `text_vert.h` declaring `text_vert_spv`. Disambiguating by directory is
// not the fix: the SYMBOL would still collide the moment both headers
// reached one translation unit, and the naming rule is what consumers write
// `#include` lines against.
//
// Measured before this check existed: ninja caught it -- `multiple rules
// generate .../text_vert.h` -- so it was never silent. What it did not do
// is name the two SHADERS, say which rule produced them, or state the way
// out; and it arrives as a graph-loading failure rather than as this rule's
// refusal. A project with one shader per stage never meets it, which is why
// it survived: a graphics project organising shaders by purpose is the
// first to have two.
{
std::map<std::string, std::string> seen; // output path -> first source
for (auto const& src : shaders) {
const std::filesystem::path p(src);
const auto stage = stage_of(p.extension().string());
if (stage.empty()) continue; // reported below, per source
// THE DIRECTORY IS PART OF THE NAME NOW, SO THIS FIRES LESS OFTEN.
//
// Before the surface existed, every shader's header and symbol came
// from its stem alone, so `a/scale.comp` and `b/scale.comp` collided
// and had to be refused. Both now land in their own namespace and
// their own subdirectory of the generated tree, so the refusal is
// for what it was always about: two shaders that are genuinely the
// same name in the same place.
std::string key;
for (auto const& seg : namespace_of(src, baseDir)) key += seg + "/";
key += p.stem().string() + "_" + std::string(stage);
auto [it, fresh] = seen.try_emplace(key, src);
if (!fresh) {
std::cerr << std::format("mcpp.rules.spirv: two shaders map to one output.\n"
" {}\n"
" {}\n"
" both produce `{}.h` declaring `{}`. The shader's directory below\n"
" `{}` is part of the name, so this is two shaders with one name in\n"
" one directory rather than two directories sharing a stem.\n"
" fix: rename one of them, or compile only one.",
it->second, src, key, symbol_of(p.stem().string(), stage),
baseDir.empty() ? std::string("the package root") : baseDir) << '\n';
return false;
}
}
}
// Collected while the actions are submitted and handed to
// `mcpp.plugins.surface` afterwards, so the declarations a consumer reads
// are written once for the whole group rather than once per shader.
std::vector<mcpp::plugins::surface::item> items;
for (auto const& src : shaders) {
const std::filesystem::path p(src);
const auto stage = stage_of(p.extension().string());
if (stage.empty()) {
std::cerr << std::format("mcpp.rules.spirv: {} has no shader stage. Both compilers derive the stage "
"from the extension; rename it to one of .comp .vert .frag .geom .tesc "
".tese .mesh .task .rgen .rint .rahit .rchit .rmiss .rcall", src) << '\n';
return false;
}
// The generated tree mirrors the shader tree below the base directory,
// so two shaders sharing a stem in different directories produce
// different files as well as different namespaces.
const auto ns = namespace_of(src, baseDir);
// THE ARRAY'S NAME CARRIES THE DIRECTORY, AND IT HAS TO.
//
// One translation unit includes every generated data header, so two
// shaders sharing a stem would declare one name twice. That is not
// caught as a redefinition, which is what makes it worth a comment:
// GCC's `#pragma once` treats two files with the same size and the same
// content as the same file, so two identical headers -- which is exactly
// what the same shader in two directories produces -- SILENTLY collapse
// to one, and both accessors return the same array. Measured on a
// fixture with `shaders/a/scale.comp` and `shaders/b/scale.comp`: the
// program printed the right magic number twice and the two pointers
// were equal.
//
// With no subdirectory the namespace is empty and the name is what it
// has always been, so nothing an existing project generated changes.
const auto sym = symbol_of(ns, p.stem().string(), stage);
auto dir = std::filesystem::path(gen);
for (auto const& seg : ns) dir /= seg;
std::filesystem::create_directories(dir, ec);
const auto base = (dir / (p.stem().string() + "_" + std::string(stage))).string();
const auto header = base + ".h";
// Declared here rather than beside the action below, because the item
// recorded a few lines down names it: the surface needs the payload's
// path to write `.incbin`, and it writes that before any action runs.
const std::string spv = base + ".spv";
// As an `#include` writes it: relative to `gen`, which is the directory
// this rule puts on the include path.
std::string headerRel;
for (auto const& seg : ns) headerRel += seg + "/";
headerRel += p.stem().string() + "_" + std::string(stage) + ".h";
// WHERE A SIDECAR IS FOUND AT RUN TIME, AND WHAT THAT COSTS.
//
// The generated accessor opens this path relative to the WORKING
// DIRECTORY, so it is written relative to the package root -- which is
// where `mcpp run` starts the program, and where a project doing shader
// hot-reload runs it from. A program started from anywhere else finds
// nothing, and that is the property that keeps this storage off the
// default rather than a defect in it: the payload is not in the
// artifact, so something outside the artifact has to be true.
const auto sidecarName =
std::filesystem::path(spv).lexically_relative(root).generic_string();
items.push_back({ .identifier = p.stem().string() + "_" + std::string(stage),
.name_space = ns,
.data_header = headerRel,
.data_symbol = sym,
.payload_path = spv,
.sidecar_name = sidecarName });
const auto input = std::filesystem::path(src).is_absolute()
? src : root + "/" + src;
// `id` and `description` are raw pointers the action reads at
// `submit()`; `arg`/`input`/`output` copy, these two do not. Held in
// named strings for the life of the statement that submits.
const std::string id = "spirv:" + src;
const std::string desc = std::string(cc.name()) + " " + src;
// ONE SHAPE FOR BOTH ROUTES: the compiler writes `<base>.inc` and this
// rule writes `<base>.h` around it. glslang used to write the header
// itself, which made the two routes' headers differ in whether they
// could be included first -- see `write_header`.
const bool embedAsSource =
opt.storage == mcpp::plugins::surface::storage::header;
const std::string inc = base + ".inc";
const std::string output = embedAsSource ? inc : spv;
// The wrapper header exists only to make the compiler's C output a
// translation unit. The other two storages never read a header, so
// writing one would leave a file nothing includes.
if (embedAsSource && !write_header(header, inc, sym, cc.kind)) return false;
mcpp::action a;
a.id = id.c_str();
a.role = "source"; // a header: ordered before compilation
a.description = desc.c_str();
a.arg(cc.path.c_str());
if (cc.kind == flavour::glslc) {
// glslc spells it as one token; glslang takes a separate argument.
a.arg(("--target-env=" + env).c_str());
} else {
a.arg("-V");
a.arg("--target-env"); a.arg(env.c_str());
}
// The stage, in each compiler's own spelling of the same idea. glslc
// accepts glslang's short names (`comp`, `vert`, ...) so the table
// above serves both -- but its `-S` means "emit assembly", so passing
// glslang's flag to it would produce a text file the program then
// includes as if it were data.
if (cc.kind == flavour::glslc) {
a.arg(("-fshader-stage=" + std::string(stage)).c_str());
} else {
a.arg("-S"); a.arg(std::string(stage).c_str());
}
if (optimize) a.arg(cc.kind == flavour::glslc ? "-O" : "-Os");
for (auto const& d : opt.defines) a.arg(("-D" + d).c_str());
for (auto const& i : opt.includes)
a.arg(("-I" + (std::filesystem::path(i).is_absolute() ? i : root + "/" + i)).c_str());
if (embedAsSource) {
if (cc.kind == flavour::glslc) {
// `-mfmt=c` is the initialiser list; the declaration around it
// was written above.
a.arg("-mfmt=c");
} else {
// `-x --vn` is what makes glslang's output a C declaration
// rather than a binary: a `const uint32_t <sym>[]` the program
// includes.
a.arg("-x");
a.arg("--vn"); a.arg(sym.c_str());
}
}
// Under `object` and `sidecar` neither flag is passed, so both
// compilers write the same thing: a bare SPIR-V module. The one place
// the two flavours differed disappears with the storage that needed it.
a.arg("-o"); a.arg(output.c_str());
// WHAT THE SHADER `#include`s, WHICH ONLY THE COMPILER CAN KNOW.
//
// `a.input()` below names the `.comp` and nothing else, and it is fixed
// when this program runs, before the compiler has read a line. A shader that includes a
// `.glsl` therefore had no edge to that file: editing it rebuilt
// nothing and the build stayed green over a stale SPIR-V module. Both
// compilers already compute the answer while parsing and will write it
// out; the field to receive it arrived in mcpp 2026.9.7.1.
//
// Two spellings of one idea, as everywhere else in this rule.
const std::string dep = output + ".d";
if (cc.kind == flavour::glslc) {
a.arg("-MD"); a.arg("-MF"); a.arg(dep.c_str());
} else {
a.arg("--depfile"); a.arg(dep.c_str());
}
a.depfile = dep.c_str();
a.arg(input.c_str());
a.input(input.c_str());
a.output(output.c_str());
a.submit();
}
// The include path carries the generated data headers so the generated
// implementation can reach them. It is not how a consumer reaches a shader:
// that is the surface below, and no consumer writes one of these names.
mcpp::include_dir(gen.c_str());
// ── What a consumer names ────────────────────────────────────────────────
mcpp::plugins::surface::options so;
so.surface = opt.surface;
so.store = opt.storage;
so.elem = mcpp::plugins::surface::element::word32;
so.module_name = moduleName;
so.out_dir = gen;
so.produced_by = "mcpp.rules.spirv";
// Answered here, not read there: `mcpp.plugins.surface` compiles into a
// plain binary as well as into this build program, so it takes its inputs.
so.target_os = mcpp::target_os();
so.has_gas_assembler = std::string_view(mcpp::compiler()) != "msvc";
// Generated and declared by whichever route the storage requires: plan
// time for `header` and `sidecar`, an action for `object`. See
// `mcpp::plugins::surface_for` for why that is a property of the storage
// rather than a choice this rule makes.
const auto out = mcpp::plugins::surface_for(items, so);
if (!out.ok) return false;
// THE DEGRADATION IS REPORTED HERE BECAUSE THE GENERATOR CANNOT REPORT IT.
//
// `mcpp::warning` is a build-program channel and `mcpp.plugins.surface`
// deliberately has none: it compiles into a plain binary as well. So the
// generator states the storage it actually used and the caller says so.
// Whoever sets `options::store` owns this: leaving it out would turn a
// stated fallback into a silent one, which is the shape this whole round
// is about.
if (out.files.store != so.store)
mcpp::warning("mcpp.rules.spirv: object storage needs a GAS assembler and this "
"toolchain has none; the payload is compiled in as generated source "
"instead. The declarations a consumer sees are unchanged.");
// Which collection produced these actions. The version was `0.1.1` while
// the package was `0.2.6` for as long as nothing read it; a fact is a
// reader, and a stale constant is now visible in every build log.
mcpp::fact("mcpp.plugins", std::string(mcpp::plugins::version).c_str());
return true;
}
// The whole manifest's worth: the shaders the constrained glob routed here.
// A build that names no accelerator has none, and the seam's CPU side carries
// the program — the same shape `mcpp::rules::cuda::compile()` has, for the same
// reason.
inline bool compile(options opt = {}) {
if (!*mcpp::accel()) return true;
// Several rules in one build program is the ordinary shape for a project
// with several backends, and each is called unconditionally -- the build
// program cannot know which backends this build named without parsing
// `accel` itself, which is what the rule already does. A rule whose
// backend this build does not name has nothing to do, and that is not a
// mistake and must not be reported as one.
if (!parse_target(mcpp::accel()).present) return true;
const auto shaders = device_shaders();
if (shaders.empty()) {
mcpp::warning("[build] accel names vulkan but no constrained glob matched a "
"shader; nothing was compiled for it");
return true;
}
return compile(std::span<const std::string>(shaders), std::move(opt));
}
} // namespace mcpp::rules::spirv