Test driven development is a software engineering philosophy - is isn't the only one (see List of software development philosophies - Wikipedia).
Clean Architecture: A Craftsman's Guide to Software Structure and Design - Robert C. Martin
Why most unit testing is a waste
- tests can tell you if you new code breaks anything
- tests make you write better code
- tests help others understand your code
A test suite gives you verification that functions are not broken
- check for correctness by failing to show incorrectness
It doesn't prove that everything is perfect!
Tests force you to decompose code into functions
- code that is difficult to test may be too tightly coupled
We must design the function so it can be tested
- think about what kinds of data structures being used - favour simple (i.e. JSON) so that a test stub can be used
- if the code is hard to test because of too many use cases, this suggests the function is doing too much
Test also force you to think about dependencies (you shouldn't need to run the database to test the UI)
Tests also act as a form of documentation - executable documentation that a computer can run (and other developers can read)
Test code is as important as your source code
Fast
Easy to run
Automatically run (pre commit git hook)
Deterministic
Unit test = tests a function/class in isolation
Integration test = several functions/classes
System test = entire system mimicking how a user would use the system
Write the test before you write the function
Check that the test fails
Write the function until the test passes
A lot of data science work is research - writing tests may not be justified
Testing of data science code often requires creativity