Skip to main content

Ctrl+K

Megatron Core

GitHub

Megatron Core

GitHub

Table of Contents

About Megatron Core

Overview
Release Notes and Roadmaps

Get Started

Installation
Your First Training Run

Basic Usage

Data Preparation
Training Examples
Parallelism Strategies Guide

Supported Models

Supported Models

Advanced Features

Deterministic Training
Mixture of Experts
Megatron-FSDP
Distributed Optimizer
Weighted Checkpoint Merge
Optimizer CPU Offload
Fine-Grained Activation Offloading
Data Loading at Scale
Megatron Energon
Megatron RL
Tokenizers
GDN Elementwise Fusion

Inference

Inference User Guide

Observability

Observability Overview
Configuration
Span Groups
Metrics
Pipeline-Parallel Trace Correlation
Extending Instrumentation

Developer Guide

Contributing to Megatron-LM
How to Submit a PR
Oncall Overview
Generating Docs Locally

API Reference

API Guide
API Reference
- core

Resources

Migrate from GPTModel to HybridModel
Discussions

API Reference
core
core.ops
core.ops.attention
core.ops.attention.csa

Is this page helpful?

`core.ops.attention.csa`#

Attention operation implementations.

Submodules#

core.ops.attention.csa.output_rope

previous

core.ops.attention

next

core.ops.attention.csa.output_rope

On this page

Submodules

Privacy Policy | Your Privacy Choices | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2026, NVIDIA Corporation.