ASDF is a huge improvement over system definition facilities that had preceded it. I always found mk:defsystem to be a bit of a monstrosity. But one of the great benefits I had expected from ASDF, extensibility, leaves a lot to be desired.
I have, after a long break, returned to integrating FReT into ASDF. I want to define an ASDF operation, along with other instrumentation, so that one can execute a FReT test suite directly through ASDF. For example, after loading FReT, one should be able to run:
(asdf:operate 'fret:fret-op 'fret-test)
This should execute the FReT test suites.
ASDF does not make this easy. ASDF does not perform any operations on components other than files. So you can't really ever instrument an arbitrary component to call an :after method. If you're loading a system, there is no way to know when you're done loading. So if I have a test suite defined on a component, and I want to test that component, I can't know when to run the test suite.
I found a way around that problem. I defined a virtual FReT source file that is added after the component is defined that depends on all other components in the system to test. I wanted to add the virtual file in the :after of initialize-instance of the fret::component class, which inherits from asdf:component. I was thwarted again. ASDF does not really do anything interesting in its make-instance method, only in its reinitialize-instance. The components however aren't even set here, but rather through a setf.
The manner in which I've had to implement this new operation seems quite convoluted. With a bit of forethought I imagine this could have been greatly simplified. For now I'm happy to have everything working.