Reply to post:

'It will go wrong. There's no question of time... on safety or security side'

The Mole

Disassembling the binary only gives you the instruction sequencing if it is single threaded application (and not even that if there are many loops and branches dependent upon external inputs). In a multi-threaded or distributed application there is generally no way to enumerate all possible interleaved sequencing of events, particularly when those events may include hardware operation and error scenarios (such as patterns of packet loss).

Carefully choosing a range of inputs is hard, very hard, how do you determine in a complex system that you have selected enough variation, particularly taking into account all possible fault conditions and event sequencing. In a complex system just because input A gives output B when you ran it in test on your perfect network at 12:32pm doesn't prove that it will give output B on a real laggy network at 12:00am. You can't test all possible scenarios so you have to narrow down the set, based on the understanding of the code and platform and where the risk is perceived to be, of course with any risk based approach sometimes you get unlucky.

If code inspection was easy then this obviously means that code that has been reviewed won't introduce any bugs and work perfectly in the system. Experience shows even with thorough code review and test definition and multi layers of validations bugs still get through to the customer.

The simple fact is most computer software is too complex for a single person to fully maintain a mental model of how it works and all the possible interactions and knock on effects of changes. Modularization and clear specification helps mitigate some of this, though that can actually sometimes be where the security or resilience holes appear.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon