Understanding Latent Dirichlet Allocation (LDA) — A Data Scientist’s Guide (Part 2) | by Louis Chan | Feb, 2024

LDA Convergence Explained with a Dog Pedigree Model

“What if my a priori understanding of dog breed group distribution is inaccurate? Is my LDA model doomed?”

My wife asked.

Welcome back to part 2 of the series, where I share my journey of explaining LDA to my wife. In the previous blog post, we discussed how LDA works and how it can be understood as a dog pedigree model.

This time around, let’s dive into the iterative fitting process of LDA!

Part 1 (link):

  • How does LDA work?
  • How to explain LDA to a non-technical person?

Part 2 (We are here now!):

  • How does LDA improve iteratively?
  • How does LDA converge?
  • Bonus: Get your LDA cheatsheet here!

Part 3:

  • When to use LDA & when not to?
  • How can we use it in Python?
  • What are the alternatives & variants to LDAs (excluding LLMs)?

Let’s get started.

If you have not read part 1 of the series, I strongly encourage you to read it first, as we will build on that understanding.

Quick Recap from Part 1

Source link


gaitQ and machineMD secure million dollar research grant to monitor Parkinson’s development in UK and Switzerland

Oxford-based medical technology start-up gaitQ and Swiss medical device company machineMD have announced the joint award of a million dollar research grant from Innovate UK and Innosuisse to enable the collection and analysis of critical movement data from people with Parkinson’s (PwP). The grant will fund an 18-month research project that will record movement data […]

Read More

Take-Two plans to lay off 5 percent of its employees by the end of 2024

Take-Two Interactive plans to lay off 5 percent of its workforce, or about 600 employees, by the end of the year, as reported in an SEC filing Tuesday. The studio is also canceling several in-development projects. These moves are expected to cost $160 million to $200 million to implement, and should result in $165 million […]

Read More

10 tips to avoid planting AI timebombs in your organization

At the recent HIMSS Global Health Conference & Exhibition in Orlando, I delivered a talk focused on protecting against some of the pitfalls of artificial intelligence in healthcare. The objective was to encourage healthcare professionals to think deeply about the realities of AI transformation, while providing them with real-world examples of how to proceed safely […]

Read More