کتاب کار با مدل‌های زبانی بزرگ

مقدمه‌ای بر مدل‌های زبانی بزرگ

What Is Language AI? • A Recent History of Language AI • Representing Language as a Bag-of-Words • Better Representations with Dense Vector Embeddings • Types of Embeddings • Encoding and Decoding Context with Attention • Attention Is All You Need • Representation Models: Encoder-Only Models • Generative Models: Decoder-Only Models • The Year of Generative AI • The Moving Definition of a “Large Language Model” • The Training Paradigm of Large Language Models • Large Language Model Applications: What Makes Them So Useful? • Responsible LLM Development and Usage • Limited Resources Are All You Need • Interfacing with Large Language Models • Proprietary, Private Models • Open Models • Open Source Frameworks • Generating Your First Text • Summary

توکن‌ها و تعبیه‌ها

LLM Tokenization • How Tokenizers Prepare the Inputs to the Language Model • Downloading and Running an LLM • How Does the Tokenizer Break Down Text? • Word Versus Subword Versus Character Versus Byte Tokens • Comparing Trained LLM Tokenizers • Tokenizer Properties • Token Embeddings • A Language Model Holds Embeddings for the Vocabulary of Its Tokenizer • Creating Contextualized Word Embeddings with Language Models • Text Embeddings (for Sentences and Whole Documents) • Word Embeddings Beyond LLMs • Using pretrained Word Embeddings • The Word2vec Algorithm and Contrastive Training • Embeddings for Recommendation Systems • Recommending Songs by Embeddings • Training a Song Embedding Model • Summary

نگاهی به درون مدل‌های زبانی بزرگ

An Overview of Transformer Models • The Inputs and Outputs of a Trained Transformer LLM • The Components of the Forward Pass • Choosing a Single Token from the Probability Distribution (Sampling/Decoding) • Parallel Token Processing and Context Size • Speeding Up Generation by Caching Keys and Values • Inside the Transformer Block • Recent Improvements to the Transformer Architecture • More Efficient Attention • The Transformer Block • Positional Embeddings (RoPE) • Other Architectural Experiments and Improvements • Summary

طبقه‌بندی متن

The Sentiment of Movie Reviews • Text Classification with Representation Models • Model Selection • Using a Task-Specific Model • Classification Tasks That Leverage Embeddings • Supervised Classification • What If We Do Not Have Labeled Data? • Text Classification with Generative Models • Using the Text-to-Text Transfer Transformer • ChatGPT for Classification • Summary

خوشه‌بندی متن و مدل‌سازی موضوعی

ArXiv’s Articles: Computation and Language • A Common Pipeline for Text Clustering • Embedding Documents • Reducing the Dimensionality of Embeddings • Cluster the Reduced Embeddings • Inspecting the Clusters • From Text Clustering to Topic Modeling • BERTopic: A Modular Topic Modeling Framework • Adding a Special Lego Block • The Text Generation Lego Block • Summary

مهندسی پرامپت

Using Text Generation Models • Choosing a Text Generation Model • Loading a Text Generation Model • Controlling Model Output • Intro to Prompt Engineering • The Basic Ingredients of a Prompt • Instruction-Based Prompting • Advanced Prompt Engineering • The Potential Complexity of a Prompt • In-Context Learning: Providing Examples • Chain Prompting: Breaking up the Problem • Reasoning with Generative Models • Chain-of-Thought: Think Before Answering • Self-Consistency: Sampling Outputs • Tree-of-Thought: Exploring Intermediate Steps • Output Verification • Providing Examples • Grammar: Constrained Sampling • Summary

تکنیک‌ها و ابزارهای پیشرفته تولید متن

Model I/O: Loading Quantized Models with LangChain • Chains: Extending the Capabilities of LLMs • A Single Link in the Chain: Prompt Template • A Chain with Multiple Prompts • Memory: Helping LLMs to Remember Conversations • Conversation Buffer • Windowed Conversation Buffer • Conversation Summary • Agents: Creating a System of LLMs • The Driving Power Behind Agents: Step-by-step Reasoning • ReAct in LangChain • Summary

جستجوی معنایی و تولید تقویت‌شده با بازیابی

Overview of Semantic Search and RAG • Semantic Search with Language Models • Dense Retrieval • Reranking • Retrieval Evaluation Metrics • Retrieval-Augmented Generation (RAG) • From Search to RAG • Example: Grounded Generation with an LLM API • Example: RAG with Local Models • Advanced RAG Techniques • RAG Evaluation • Summary

مدل‌های زبانی بزرگ چندوجهی

Transformers for Vision • Multimodal Embedding Models • CLIP: Connecting Text and Images • How Can CLIP Generate Multimodal Embeddings? • OpenCLIP • Making Text Generation Models Multimodal • BLIP-2: Bridging the Modality Gap • Preprocessing Multimodal Inputs • Use Case 1: Image Captioning • Use Case 2: Multimodal Chat-Based Prompting • Summary

ایجاد مدل‌های تعبیه متن

Embedding Models • What Is Contrastive Learning? • SBERT • Creating an Embedding Model • Generating Contrastive Examples • Train Model • In-Depth Evaluation • Loss Functions • Fine-Tuning an Embedding Model • Supervised • Augmented SBERT • Unsupervised Learning • Transformer-Based Sequential Denoising Auto-Encoder • Using TSDAE for Domain Adaptation • Summary

تنظیم دقیق مدل‌های نمایش برای طبقه‌بندی

Supervised Classification • Fine-Tuning a Pretrained BERT Model • Freezing Layers • Few-Shot Classification • SetFit: Efficient Fine-Tuning with Few Training Examples • Fine-Tuning for Few-Shot Classification • Continued Pretraining with Masked Language Modeling • Named-Entity Recognition • Preparing Data for Named-Entity Recognition • Fine-Tuning for Named-Entity Recognition • Summary

تنظیم دقیق مدل‌های تولید

The Three LLM Training Steps: Pretraining, Supervised Fine-Tuning, and Preference Tuning • Supervised Fine-Tuning (SFT) • Full Fine-Tuning • Parameter-Efficient Fine-Tuning (PEFT) • Instruction Tuning with QLoRA • Templating Instruction Data • Model Quantization • LoRA Configuration • Training Configuration • Training • Merge Weights • Evaluating Generative Models • Word-Level Metrics • Benchmarks • Leaderboards • Automated Evaluation • Human Evaluation • Preference-Tuning / Alignment / RLHF • Automating Preference Evaluation Using Reward Models • The Inputs and Outputs of a Reward Model • Training a Reward Model • Training No Reward Model • Preference Tuning with DPO • Templating Alignment Data • Model Quantization • Training Configuration • Training • Summary