Transfer Learning and Federated Learning: A Primer - Part I

Advanced technologies abound in the financial industry - but we rarely see an exploration of the different kinds of machine learning, or how they work for compliance. In this two-part blog series, we’ll explore what transfer learning and federated learning are, how they work for compliance practices, and how you can use them together for an ironclad compliance practice. 

What is Basic Machine Learning? 

Traditional machine learning, or basic machine learning, is what we call “single-task learning”. With single-task learning, an algorithm is informed by a dataset and that information is applied to a specific task. With traditional machine learning, the knowledge is not accumulated from task to task, and it does not consider past lessons learned. 

In practice, traditional machine learning can be illustrated in this way. An algorithm learns how to attribute the features of something - for example, when you search Google for "cat", the algorithm has to decide what "is cat" and what is "not cat". That's how an algorithm works in its most basic form. As an algorithm learns, it is then applied to a dataset where it indicates which items are what it learned, and which are not (in this case, cats or not cats). Now, to make it more applicable for the financial industry, change "cats" in this example to any kind of risk. Understanding basic machine learning highlights how advances such as transfer and federated learning will benefit your compliance practice through improved risk detection.

What is Transfer Learning? 

Transfer learning advances on the basic machine learning above. In transfer learning, an algorithm learns from a data set, or a “problem”, and then with that stored knowledge, it applies those lessons to new data. As this process happens, the algorithm continues to learn from each problem, moving from dataset to dataset. Unlike basic machine learning above, transfer learning continues to learn on each individual dataset it encounters, accumulating these lessons to improve precision. 

The beauty of transfer learning is that with each deployment of the algorithm, its knowledge of the problem becomes more precise. As seen in the image above, this is exactly how GOST (Giant Oak Search Technology) works. Algorithms are trained on data sets- identifying money laundering, terrorism, or corruption- and then as they are deployed on new problems, they continue to learn and refine. 

What is Federated Learning? 

As opposed to transfer learning, federated learning is a type of machine learning that trains an algorithm on multiple devices, or sets of data, without actually exchanging the data itself. This innovation came about because transfer learning requires the sharing of data in order to advance - as its name suggests. That presents a problem, however, for a world where data shouldn’t - and often can’t- be shared. How do you create precise algorithms that solve problems efficiently without sharing data? 

Enter federated learning. As seen in the diagram below, federated learning begins with an algorithm that passes from dataset to dataset and learns to recognize the features of a specific behavior or attribute with each data set it visits. With every new data set, the algorithm becomes more precise, but the data itself remains in its secured environment. Imagine that the algorithm is a bee and the data sets are flowers. As the bee travels from flower to flower, it gathers pollen (knowledge), but the flower (data) remains in its original location.

The Upshot

Machine learning capabilities have advanced exponentially over the past decades, and capabilities now offer unprecedented opportunities for practices to benefit from well-trained algorithms and can process large amounts of data quickly. In order to explore these applications, however, it’s important to understand the different mechanisms that make it possible. Transfer learning and federated learning are the future of the compliance space. In the next blog post, we’ll use this basis to explore how both together can result in an ironclad compliance practice.