AnglE 📐

📢 Train/Infer Powerful Sentence Embeddings with AnglE.

This library is from the paper Angle-optimized Text Embeddings . It allows you to train state-of-the-art BERT/LLM-based sentence embeddings with just a few lines of code. AnglE is also a general sentence embedding inference framework, allowing for infering a variety of transformer-based sentence embeddings.

✨ Features

Loss:

1. 📐 AnglE loss

2. ⚖ Contrastive loss

3. 📏 CoSENT loss

4. ☕️ Espresso loss (previously known as 2DMSE)

Backbones:

1. BERT-based models (BERT, RoBERTa, ELECTRA, ALBERT, etc.)

2. LLM-based models (LLaMA, Mistral, Qwen, etc.)

3. Bi-directional LLM-based models (LLaMA, Mistral, Qwen, OpenELMo, etc.. refer to: https://github.com/WhereIsAI/BiLLM)

Training:

1. Single-GPU training

2. Multi-GPU training

Contents:

• 🚀 Quick Start
  • ⬇️ Installation
• 🔍 Inference
  • 1️⃣ BERT-based Models
  • 2️⃣ LLM-based Models
  • 3️⃣ BiLLM-based Models
  • 4️⃣ Espresso/Matryoshka Models
  • ⚡ Batch Inference
  • 📚 Next Steps
• ⬇️ Installation
• 📦 Installation Methods
  • Method 1: Using uv (Recommended)
  • Method 2: Manual Installation
• 👨‍🏫 Tutorial
  • 📋 Overview
  • Step 1: Data Preparation
  • Step 2: Train the Model
  • Step 3: Evaluate the Model
  • Step 4: Use the Model
  • 🎓 Summary
• 🏛️ Official Pretrained Models
  • BERT-based models:
  • LLM-based models:
• 🚂 Training and Finetuning
  • 🗂️ Data Preparation
  • 🎯 Training Methods
  • ⚙️ Configuration & Hyperparameters
• 🎯 Evaluation
  • 📊 Overview
  • 🎯 Spearman and Pearson Correlation
  • 📚 Next Steps
• 🫡 Citation

APIs:

• API Reference