LITTLE DATA FACTORY
The demand for synthetic data is huge - according to Gartner, synthetic data will completely overshadow real data in AI models by 2030.
Collecting data in the real world can be hard and very expensive, and even sometimes impossible because of safety or physical constraints. But synthetic generation is not limited in the same way, allowing plentiful data to be created across many different scenarios. One particular target use for synthetic data creation is for training machine learning (ML) models. These require vast amounts of data, ideally labelled, and obtaining such data in sufficient quantities can be a limiting factor to ML development.
Defence is a sector that can benefit from synthetic data, and Dstl were interested in how synthetic multimodal data might generated routinely, along the lines of a “Data Factory”. But first they wanted to prove the concept by developing a first implementation and overcome some particular technical challenges. We carried out this project under Serapis Lot 6 “Understand”, which is managed by Frazer-Nash Consultancy for Dstl, and were supported by SVGC and its subject matter experts in military operations.
There were three key challenges relating to synthetic data creation that the client posed to us, and finding solutions to each of them was crucial to deciding whether a data factory could succeed. The first was being able to create diverse multi-modal datasets that were coherent and consistent with one another – this is important as ML agents may want to combine different sources of data when making its predictions. The second was being able to seed events, for example requiring a particular event happens at a particular time, which sounds easy but because of the interdependence of events and the need for consistency and realism it requires real care. The final one was being able to give input to the Data Factory as a high-level storyline created by a human, which it then interprets and converts to a detailed machine-readable input for the model to use.
Send us an email, to discuss a new project.
We’re a team of innovators who are excited about unique ideas and help companies to create amazing solutions.