Staging environment

LLM Engineering - Foundations to SLMs

·

5 Weeks

·

Cohort-based Course

Master language models and embedding models through training, fine-tuning, aligning, distilling, and merging transformer architectures!

Course overview

Understand LLMs from first principles; build custom Small Language Models (SLMs)

🧑‍💻 Language Model Engineering refers to the evolving set of best practices for training, fine-tuning, and aligning LLMs to optimize their use and function to balance performance with efficiency.


These best practices have formed the basis for the LLMs, or Large Language Models (a.k.a. Foundation Models) and Small Language Models (SLMs) of today.


🤖 Whether you’re looking at OpenAI’s GPT series, Anthropic Claude, Grok, Google Gemini, Mistral, or any other model provider, the core underlying transformer architectures are similar, as are the training and tuning methods.


When chasing after high scores on LLM benchmarks or creating state-of-the-art SLMs, techniques like Model Merging and Distillation are important as well.


🏫 This course will provide you with the foundational concepts and code to train, fine-tune, and align state-of-the-art SLMs and LLMs using industry-standard and emerging approaches from the open-source edge heading into 2025.


🤓 Become the expert in your organization on all things training (pretraining, post-training), fine-tuning (supervised fine-tuning, instruction tuning, chat tuning, etc.), alignment (PPO, DPO, etc.), Small Language Models (SLMs), Model Merging, Distillation, and more!

Who is this course for

01

Data scientists, researchers, or engineers experienced with classic Machine Learning who want to build custom Small Language Models (SLMs).

02

Product managers who need to understand the core concepts and code behind custom SLMs to effectively lead their product teams.

03

Stakeholders who aim to train, fine-tune, or align proprietary SLMs for their customers in 2025.

By the end of this course, you will be able to:

🦾 Build, train, and evaluate Language Models using transformer variants (GPT, BERT, BART)


🧐 Calculate self-attention and understand the latest implementations (Flash Attention, FA2)

🔠 Demystify embedding layers, embedding representations, pre-trained vs. learned embeddings, ROPE, and more!

🪙 Decode embedding space representations for optimal next-token prediction

🔡 Build, train, and evaluate embedding models (like those in 🤗 Sentence Transformers)

🚇 Complete unsupervised and continued pretraining of LLMs and SLMs from scratch

🚉 Fine-tune pre-trained LMs for instruction-following, chat, and more via parameter-efficient methods

🛤️ Align LMs to balance helpfulness with harmlessness and other criteria (RLXF, DPO)

🚀 Explore frontiers of Language Models (Mixture-of- approaches, Model Merging, alternative fine-tuning, and more!)

What’s included

Live sessions

Learn directly from "Dr. Greg" Loughnane & Chris "The Wiz 🪄" Alexiuk in a real-time, interactive format.

Lifetime access

Go back to course content and recordings whenever you need to.

Community of peers

Stay accountable and share insights with like-minded professionals.

Certificate of completion

Share your new skills with your employer or on LinkedIn.

Maven Guarantee

This course is backed by the Maven Guarantee. Students are eligible for a full refund up until the halfway point of the course.

Course syllabus

78 lessons • 8 projects

Week 1

Feb 11—Feb 16

    🏈 Cohort Kick-Off!

    6 items

    🤖 The Transformer Paper: Attention is All You Need

    8 items

Week 2

Feb 17—Feb 23

    🧐 Attention

    6 items

    🔠 Embeddings

    10 items

Week 3

Feb 24—Mar 2

    🪙 Next-Token Prediction

    9 items

    🔡 Embedding Models

    9 items

Week 4

Mar 3—Mar 9

    🚇 Pretraining

    10 items

    🚉 Fine Tuning

    11 items

Week 5

Mar 10—Mar 13

    🛤️ Alignment

    10 items

    🌌 LLM Engineering Frontiers

    7 items

What students are saying

LLM Engineering vs. AI Engineering: What's the Difference?

Prerequisites

A background in fundamental Machine Learning and Deep Learning

Understanding supervised learning, unsupervised learning, and neural network architectures is required. Introductory NLP and Computer Vision knowledge is encouraged. Not sure where to start? Read this.

A ability to program in Python within a Jupyter Notebook Environment

Understand basic Python syntax and constructs. You should be comfortable training and evaluating simple ML & DL models using test, train, and dev sets. Not sure where to start? Look here.

Free resource

The LLM Engineering Onramp

You’ll find brief introductions to core concepts we’ll cover in more depth in the class - this is meant to get the learning juices flowing with bite-sized training materials from Dr. Greg and The Wiz.

Start learning for free now!

Free resource

📅 Detailed Schedule!

Understand how everything comes together in the course to provide a holistic overview of the how LLMs are engineered.


Get all the details about the assignments, associated papers, and key concepts you'll learn!

Send me the deets ✌️

Meet your instructors

"Dr. Greg" Loughnane

"Dr. Greg" Loughnane

Co-Founder & CEO @ AI Makerspace

In 2023, "The Wiz 🪄" a.k.a. "The LLM Wizard 🪄" and I created the LLM Engineering: The Foundations and LLM Ops: LLMs in Production courses. In 2024, we launched The AI Engineering Bootcamp.


From 2021-2023 I led the product & curriculum team at FourthBrain (Backed by Andrew Ng's AI Fund) to build industry-leading online bootcamps in ML Engineering and ML Operations (MLOps).


Previously, I worked as an AI product manager, university professor, data science consultant, AI startup advisor, and ML researcher; TEDx & keynote speaker, lecturing since 2013.


👨‍🏫 Learn with us free on YouTube!

👨‍💼 Connect with me on LinkedIn!

Chris "The Wiz 🪄" Alexiuk

Chris "The Wiz 🪄" Alexiuk

Co-Founder & CTO @ AI Makerspace

During the day, I work as a Developer Advocate for NVIDIA. Previously, I worked with Greg at FourthBrain (Backed by Andrew Ng's AI Fund) on MLE and MLOps courses, and on a few Deeplearning.ai events!


A former founding MLE and data scientist, these days you can find me cranking out Machine Learning and LLM content!


My motto is "Build, build, build!", and I'm excited to get building with all of you!


👨‍🏫 YouTube: AI Makerspace OfficialMy Personal

👨‍💼 Connect with me on LinkedIn!

A pattern of wavy dots

Be the first to know about upcoming cohorts

LLM Engineering - Foundations to SLMs

Course schedule

4-6 hours per week

  • Class!

    Tuesdays & Thursdays, 4:00-6:00pm PT

    • Feb 11th, 13th
    • Feb 18th, 20th
    • Feb 25th, 27th
    • Mar 4th, 6th
    • Mar 11th, 13th
    • Mar 18th, 20th
  • Weekly Programming Projects

    2-4 hours per week

    Each class period, we will get hands-on with Python coding homework!

  • Office Hours

    Tuesdays and Fridays

    Dr. Greg: Thursdays @ 8 AM PT

    The Wiz 🪄: Fridays @ 3 PM PT

  • Peer Support Sessions

    Varies

    Peers supporters are here to serve as your destination group; they're available for homework help, concept deep dives, and to hang out with!


    Find them live in class, on Discord, and even in their own dedicated weekly sessions!

Build 🏗️, ship 🚢, and share 🚀 like a legend.

Build 🏗️, ship 🚢, and share 🚀 like a legend.

Hands-on learning at the LLM edge

We teach concepts AND code. Never one or the other. AI has accelerated so quickly that anyone who's been a manager the last few years has not yet seen the code for themselves.

Pair programming made fun and easy

Grow your network. Build, ship, and share with a community! Build relationships with peers and instructional staff to unlock opportunities in the years ahead.

Find fellow travelers for your journey

Join a community of like-minded AI practitioners who are all in on Generative AI, and who are heading down the same career path as you are.

Frequently Asked Questions

A pattern of wavy dots

Be the first to know about upcoming cohorts

LLM Engineering - Foundations to SLMs