Collecting both my ideas and resources from around the web on ML Debugging tools.
Show me what's going on: visibility into the system.
Ideally: cheap, always available, very reliable
Let me tweak something and tell me if it fixed the problem.
Ideally: fast, doesn't waste resources or time and proceeds from where the tweak was applied
ICLR on debugging (2019)