Infrastructure

LLM inference in C/C++

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Port of OpenAI's Whisper model in C/C++

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

A generative speech model for daily dialogue.

Cross-platform, customizable ML solutions for live and streaming media.

Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/

Find secrets with Gitleaks 🔑

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

A list of free LLM inference resources accessible via API.

Faster Whisper transcription with CTranslate2

End-to-end, code-first tutorials for building production-grade GenAI agents. From prototype to enterprise deployment.

High-performance In-browser LLM Inference Engine

The absolute trainer to light up AI agents.

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

Weaviate is an open-source vector database that stores both objects and vectors, allowing for the combination of vector search with structured filtering with the fault tolerance and scalability of a cloud-native database​.

An orchestration platform for the development, production, and observation of data assets.

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Official inference library for Mistral models

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly.

Local AI anywhere, for everyone — LLM inference, chat UI, voice, agents, workflows, RAG, and image generation. No cloud, no subscriptions.

minimax m3 free model ai model free api large language model llm 1m context window sparse attention msa architecture native multimodality computer use computer control autonomous coding assistant agentic ai framework swe bench pro software engineering workflow automation multi agent hugging face api access local llm inference

A curated collection of datasets for Large Language Models (LLMs), covering medical AI, NLP, multimodal learning, instruction tuning, reasoning, code generation, and evaluation benchmarks.