Repository navigation
Parameter dumps for configuration testing #539
Description
We have recently had two pull requests (#526, #537) where configuration options would be specified but would not make it into the C++ side of the build path. Currently, there is no way to integration-test the configuration all the way through. There are unit tests that test config file parsing and defaulting, there are tests for instantiation file generation, and there are tests for builders. However, these two bugs are both the result of one of these components not talking to another correctly. We need a way to test that a given configuration compiles to a given architecture.
I would like to see a command-line option for ChampSim that, rather than running any simulation, prints all of the properties of all components in a machine-readable format, then quits. For example:
$ ./bin/champsim --dump
CACHE cpu0_L1I.sets 512
CACHE cpu0_L1I.ways 8
(many more lines of output...)
CPU cpu0.retire_width 6
CPU cpu0.lq_size 72
(and so on...)
Here, I've used --dump as the option, but the name could be anything else. The benefit here is that the values printed are the values seen by the simulation runtime, so we are testing that the intent of the user is respected.
Once we have this, we could run any configuration files, perhaps starting with the ones in this directory, through the configure-make-run process and show that they do not break over time. I think this is especially important in the context of #424, because the inputs (configuration files) and outputs (machine-readable dump) do not change, only the process by which they are brought together.
I would love for someone to volunteer for this. This is likely an easy way to get started with contributions to ChampSim, since it requires no prior knowledge of how the insides of the system work.
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
maccoymerrell commentedon Nov 9, 2024
This is something that I think would be helpful as well.
Runtime configuration should be able to support this inherently when I get around to it, but in the meantime I think this may be possible using some sort of type wrapper around what we consider parameter values for each component. Maybe we could use the factories to help with this as well. This wrapper would save a reference to the parameter, and then when invoked for a specific component print out the names and values of all the registered parameters.
Something like
Which builds a map of the defined parameters and their references.
Then, a generic fmt format string can be made using the NAME, class_name, parameter name, and value of the reference.
The printout can be invoked using cpm.print() or something like that.
If we make it static, we could probably make it so that it only takes a single instance of champsim_parameter and thus one call to print() is needed. Everything in champsim could invoke
champsim::champsim_parameter.register<type T>(string name, &T)to register a parameter. This way, even modules could add their own custom parameters to the printout if they wanted to.
ngober commentedon Nov 9, 2024
I think there's an even simpler solution: create a
friend void print_dump()function for each of the components in question, which spits out everything that we might want, similar to howvoid print_deadlock()is now. Your registration idea holds internal references, which can break if the component is moved (#471).I think this feature should precede runtime configuration, because it provides some assurance that the
./bin/champsim --dump --config=config.jsoninvocation works exactly the same as the./config.sh config.json && make && ./bin/champsim --dump. I want some way to guarantee that no feature breaks in the migration.maccoymerrell commentedon Nov 12, 2024
I wonder if we could avoid moving things around during runtime at all, particularly in reference to runtime config. I guess one thing you could do is prevent components from receiving or giving any references until all the components have been given a fixed location. Some sort of initialize() function maybe? Ditch constructors altogether.
Do you think the right way to approach this (print_dump()) is with a champsim::component class that defines the function, and then make all the components (operables, vmem, etc...) derive from that? Or just add it to champsim::operable and add it as a separate function for vmem?
ngober commentedon Nov 12, 2024
Briefly regarding move semantics, I don't think we should assume something never needs to be moved, nor do I think we should restrict move semantics arbitrarily. If something can be movable, it should be movable.
I think it should just be enough to create a friend free function for each of the things we want to dump. We don't have that many things: cores, caches, ptws, dram, vmem, maybe channels. Inheritance doesn't make sense for this because it isn't part of the interface and there's nothing that's common to all of the components except for maybe the format string.
maccoymerrell commentedon Nov 12, 2024
I mean you could say the same thing about print_deadlock(). I am thinking for people writing code, what makes it easier to add a print_dump() for their own custom or modified component? I don't think we would be having this conversation if all parts of champsim were operables, we would just add it to the operable class and be done.
We could move all the print_deadlock(), initialize(), begin/end_phase(), warmup, and now print_dump() into the component class, and leave operable() as handling the actual operable stuff. Vmem could then inherit these functions/members even though its not an operable, which may be useful in the future.
ngober commentedon Nov 14, 2024
That makes some sense. If we go this route, I don't want to make
operableinherit fromcomponent, but ratherCACHEinherits from both. That is, a component exposes an interface to the runtime, and an operable exposes an interface to the clock bus. Actually, now that I'm typing this out I like this idea more.maccoymerrell commentedon Nov 14, 2024
Yay. I can try to do a rough implementation and see if there are any problems with this.
maccoymerrell commentedon Nov 15, 2024
I have done the component implementation. Now I am stuck on the best way to approach the print_dump() implementation.
Hardcoding the printouts for all the components seems bad, but I am not sure if there is a better way to do it.
I like the idea of a component being able to identify which of its member variables should be dumped, and then having a single function that prints out the parameters used for all components. I can't think of a good way to do it without references though.
Maybe we should be under the impression that all components' input parameters are const, so capturing a single value at the construction of the component is sufficient. Thoughts?
ngober commentedon Nov 15, 2024
I think until C++26/29 gives us a reflection syntax (I think it's coming!!!), we're stuck with manually writing one line per thing we want printed. Either we manually register all of the parameters like you suggest then print in a loop, or we manually write a dump function with one line per parameter. There just isn't a way out of it.
Capturing a single value or no, my vision for this feature is one where we run
./bin/champsim --dumpand it prints everything then quits. Const or no const, the only thing we're doing here is reading the values.