A Look at Application Testing in a Near Future

For some silly reason I was just awoken by the following thoughts about how application tests will be done in the future. I’m thinking this starts phasing in over the next 3-10 years.

  • The application will be decomposed using an automated tool to break it into the binary, the source, the network interfaces, the application-layer interfaces, and any other components / surface area.

  • The binary will be published to a harness location that allows approved tools and partners to access it in a standard way. This will include third party services, third party paid algorithms, as well as a host of free automated fuzzing algorithms.

  • The source code is published to a source code testing harness that works in a similar way. A set of free automated tools start working on it immediately, a beacon is published to the trusted consultancies that have paid for access to the test interface, and vetted individual testers will be notified as well that they can begin testing as well.

  • Network and application layer interfaces are enrolled into similar testing harnesses, which also spawn notifications to the tools, consultancies, and individuals who are partnered to test the application.

  • Submitted vulnerabilities, contracts for who is paid what based on what is found, are all handled by Ethereum, i.e., a smart-contract-based blockchain. This handles who found what, which items are duplicates, and how much was paid to find each vulnerability.

So it becomes all about the inventory of the components of the application and their arranging into a standard testing harness.

From there, tens, hundreds, or thousands of automated testing algorithms start firing to find vulnerabilities. They range from simple, rules-based systems to deep learning powered algorithms powered by terabytes of testing data for extremely similar apps—and those algorithms are updated continually.

Those algorithms are also competing with the human element, which might also have their own algorithms they can leverage as well. So human testers / researchers will be signed up for multiple contracts, using Ethereum or something similar. And when the notification goes out that an app has become available in a contract you’re a part of, you can immediately launch any of your automated tools as well as sit down to test yourself.

So imagine a binary sliding into a ship hangar, and a number of cables come from all sides and slide into their ports on the binary. 14 million people are notified that this binary is available for testing, and tens of thousands of algorithms start their work, which discovers vulnerabilities instantly and reports them to the blockchain.

Same for the remote components. The running application is slid into a similar hangar, and connectors touch the TCP/IP stack, the app layer in various ways, etc., and automated probes start instantly. The best common rulesets. The best ML/DL algorithms for that particular application type. A set of fuzzing algorithms kicks off that was just updated 12 minutes ago and is the best in the world at crashing these particular services.

For many applications that do or will have external public interfaces, notifications are also sent out to millions of independent testers simultaneously, with the only criteria being that they must be part of the Ethereum network that manages the integrity of contracts, submissions, and payment.

And then everyone goes to work on the app. All its components. Humans and algorithms battling it out. With findings streaming in over a high-integrity, high-transparency blockchain.

Insurance companies will require that applications be continuously tested using these types of harnesses, and they’ll be rated with various levels of security based on the type and duration of scrutiny the application has endured without finding additional vulnerabilities.

A number of these elements are already starting, or are already in place, but what I find most interesting about the model is breaking the app into components, using common interfaces for testing those components, and then a common contract and reporting mechanism based on the blockchain.

—

Ok, now coffee and breakfast.

Notes

  1. I say applications here, but this is really any kind of security testing. And actually, not even just security. It’s a testing and validation framework, and the testing can have multiple forms.

  2. A number of things will be needed to get this going, including some sort of mechanism for instancing. You can’t have everyone touching the same system at the same time. Especially for dynamic testing.

  3. Side note: I really dislike posts titled, “The future of”, or “In the future”. It’s cliche. It’s pompous. It’s just bad. Need to figure out better ways to convey these types of interesting conjectures / predictions of what’s to come.

Related posts: