A high-throughput and memory-efficient inference and serving engine for LLMs

A framework for efficient model inference with omni-modality models

Every Eval Ever is a shared schema and crowdsourced eval database. It defines a standardized metadata format for storing AI evaluation results — from leaderboard scrapes and research papers to local evaluation runs — so that results from different frameworks can be compared, reproduced, and reused.

Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

Redis for LLMs

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM