Large Language Models (LLMs)

Large language models are giant Transformer-based neural networks trained on massive text corpora, able to understand, generate and reason human language.

Pre-training Stage

Model learns general grammar, world knowledge and logic from billions of text tokens with next-token prediction objective.

SFT & Human Alignment

Supervised Fine-Tuning and RLHF make outputs follow human instructions, improve helpfulness, honesty and harmlessness.

Hallucination Problem

LLMs may generate plausible but false facts, citations or reasoning results, one of the core limitations in production usage.