Self-Organizing Maps - 90s Fad or Useful Tool? (Part 1)

In this post, I will explain how self-organizing maps (SOMs) work.  In the first part of this post, I'll explain the technological underpinnings of the technique.  If you're impatient and just want to get to the implementation, skip to part 2.

A few years ago I was having a discussion with a computational chemistry colleague and the topic of self-organizing maps (SOMs) came up.   My colleague remarked, "weren't SOMs one of those 90s fads, kind of like Furbys"?  While there were a lot of publications on SOMs in the early 1990s, I would argue that SOMs continue to be a useful and somewhat underappreciated technique.

What Problem Are We Trying to Solve?

In many situations in drug discovery, we want to be able to arrange a set of molecules in some sort of logical order.  This can be useful in a number of cases.
  1. Clustering.  Sometimes we want to be able to put a set of molecules into groups and select representatives from each group.  This may be the case when we only have resources to screen a subset of the compounds we have available.  In this case, the SOM performs a function similar to what we get with k-means clustering.  One advantage of the SOM is that it can provide a visual representation of the relationships between clusters. 
  2. Understanding the results of high-throughput (or other) screens.  When we run a screen, one of our first objectives is to examine the results and identify potential SAR.   One way to do this is to create a projection where similar compounds are in similar parts of space.  As we will see in subsequent sections, SOMs can provide an overview of how hits are distributed in chemical space. 
  3. Comparing compound sets.  Let's say that we have a database of compounds for fragment screening.  If we have the opportunity to add new compounds to our fragment database, we would first like to see how similar the new compounds are to our current collection.  The SOM provides a convenient method of visualizing which parts of chemical space are covered by each collection. 
How Does the SOM work? 

In order to explain SOMs we need to take a step back and take a look at what we can do with a set of colors.  Let's say that we have a random set of colors like the one shown on the left and we want to arrange these colors logically like the ones on the right.



As we all probably know, colors can be represented as a vector of three integers between 0 and 255 with the values representing the red, blue, and green components of the color.  


Since we have colors represented as vectors, we can calculate the Euclidean distance between these vectors to identify which colors are most similar.   In the example below, we calculate the distance between a vector representing the color red and vectors representing other colors.   As we can see in the figure below, red is closest to purple and farthest from green. 


Let’s say that we want to map colors into a 10x10 grid.  We can start by creating a 10x10 array of vectors.  Each vector consists of 3 elements, each of which is assigned a random value between 0 and 255.  


Now that we have a grid initialized with random vectors,  we will select a random color.  In this case, we'll select blue. 




We then calculate the Euclidean distance between our selected vector (0,0,255) and the random vectors we created above.  We then find the closest cell to the vector representing blue.  We refer to this cell as the best matching unit (BMU).



We then add some "blue" (the vector (0,0,255)) to the cells surrounding the BMU.  An exponential function is used to adjust the amount of "blue" added to surrounding cells.  We add large amounts of "blue" to adjacent cells.  A decreasing amount of blue is added to cells as we get farther away from the BMU. 

We then repeat this process of selecting a random color, identifying the closest cell to that color, and adding the selected color to the adjacent cells.  Repeating this process over thousands of cycles allows us to arrange a set of vectors so that similar vectors are close together and different vectors are far apart.  The video below shows how the SOM moves from a random set of colors to a set that is organized so that similar colors are close together. 



Remember that we are not really dealing with colors, we are simply adding vectors to create a set of basis vectors.  Any new vector can then be placed into the appropriate cluster by calculating the Euclidean distance from that vector to each basis vector.  The cell with the smallest Euclidean distance is the BMU.  

Applying the SOM to Molecules

At this point, you have probably figured out that we can use the same techniques with molecules that we just showed with colors. We can use any of a variety of methods to represent a molecule as a bit vector, where each bit represents the presence or absence of a particular substructure. 


If we're building a SOM for molecules, we begin by initializing the SOM with random vectors.  We then select a random molecule from our dataset, calculating its fingerprint and treating this fingerprint as a vector.  Once the fingerprint has been generated, we calculate the Euclidean distance from this vector to each of our random vectors.  The random vector with the smallest Euclidean distance is selected as the BMU.  The weights of the surrounding cells are then adjusted as shown above.  This process is repeated several thousand times to train the SOM.  After training, we have a set of basis vectors that can be used to cluster new molecules.  One of the great aspects of this technique is that we can train the SOM on a subset of the data.  If we have a set of 1 million molecules, we can typically train the SOM on a smaller subset (perhaps 100K) of molecules.  We can then use the basis vectors generated during training, to cluster the entire 1 million molecule database. 

That's it for the preliminaries.  If you'd like to see how this works in practice, please check out part 2 of this post.


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Comments

Post a Comment

Popular posts from this blog

Generative Molecular Design Isn't As Easy As People Make It Look

Image
I was taken aback by a recent CNBC article entitled “ Generative AI will be designing new drugs all on its own in the near future ”.  I should know better than to pay attention to AI articles in the popular press, but I feel that even scientists working in drug discovery may have a skewed perception of what generative AI can and can’t do.  To understand exactly what’s involved, it might be instructive to walk through a typical generative molecular design workflow and point out a few things.  First, these programs are far from autonomous.  Even when presented with a well-defined problem, generative algorithms produce a tremendous amount of nonsense.  Second, domain expertise is essential when sifting through the molecules produced by a generative algorithm.  Without a significant medicinal chemistry background, one can’t make sense of the results.  Third, while a few nuggets exist in the generative modeling output, a lot of work and good old-fashioned c...
Read more

AI in Drug Discovery 2023 - A Highly Opinionated Literature Review (Part I)

Image
Here’s the first part of my review of some interesting machine learning (ML) papers I read in 2023.  As with the previous editions , this shouldn’t be considered a comprehensive review.  The papers covered here reflect my research interests and biases, and I’ve certainly overlooked areas that others consider vital.  This post is pretty long, so I've split it into three parts, with parts II and III to be posted in the next couple of weeks.    I. Docking, protein structure prediction, and benchmarking II. Large Language Models, active learning, federated learning, generative models, and explainable AI III. Review articles 2023 was a bit of a mixed bag for AI in drug discovery.  Several groups reported that the deep learning methods for protein-ligand docking weren’t quite what they were initially cracked up to be.  AlphaFold2 became pervasive, and people started to investigate, with mixed success, the utility of predicted protein structures.  There...
Read more

AI in Drug Discovery - A Highly Opinionated Literature Review (Part II)

Image
Picking up where we left off in Part I , this post covers several other ML in drug discovery topics that interested me in 2023.  Some areas, like large language models, are new, and most of the work is at the proof-of-concept stage.  Others, like active learning, are more mature, and several groups are starting to explore nuances of the methods.   Here’s the structure of Part II.  4. Large Language Models 5. Active Learning 6. Federated Learning 7. Generative Models 8. Explainable AI 9. Other Stuff 4. Large Language Models The emergence of GPT-4 and ChatGPT brought considerable attention to large language models (LLMs) in 2023.  In November and December, several large pharmas held “AI Day” presentations featuring LLM applications for clinical trial data analysis. Many of these groups demonstrated the ability of LLMs to ingest large bodies of unstructured clinical data and subsequently generate tables and reports based on natural language queries.  Asid...
Read more