Layers API Reference

Core building blocks for Sparse Selective Hyper-Connections.

CayleyTransform

BatchedCayleyTransform

SparseOrthogonalMixture

TripleRoutingMatrices

FactorizedKVCache

AdaptiveRankSelector

Base Classes

BaseSHCLayer

BaseRoutingLayer

BaseCacheLayer