bigc

datatrove. Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

3FS. A high-performance distributed file system designed to address the challenges of AI training and inference workloads.

data-juicer. Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

RedPajama-Data. The RedPajama-Data repository contains code for preparing large datasets for training large language models.

DataMaster. official code for DataMaster

ds4. DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm

mlfs-book. O'Reilly book - Building Machine Learning Systems with a feature store: batch, real-time, and LLMs

claw-code. An agent-managed museum exhibit, built in Rust with Gajae-Code / LazyCodex — developed and maintained with no human intervention.

agent-matrix. AI驱动的内容矩阵管理系统，支持多个Agent自动生成内容并发布到小红书、抖音等平台

nanobot. Lightweight, open-source AI agent for your tools, chats, and workflows.

opencode. The open source coding agent.

cc-switch. A cross-platform desktop All-in-One assistant for Claude Code, Codex, OpenCode, OpenClaw, Gemini CLI & Hermes Agent. Only official website: ccswitch.io

stream-DaQ. 🦆 Stream-first data quality monitoring in Python! Learn more: https://arxiv.org/abs/2506.06147

SparkDQ. The code repository for SparkDQ, a big data quality management system.

OpenMetadata. The Open Context Layer for Data and AI , OpenMetadata is the open platform for building trusted data context and business semantics for humans, AI assistants, and agents.

unitycatalog. Open, Multi-modal Catalog for Data & AI

Modern-AI-Agents. Modern AI Agents

deep-learning-from-scratch-5. 『ゼロから作る Deep Learning ❺』(O'Reilly Japan, 2024)

qwen-code. An open-source AI coding agent that lives in your terminal.

minimind. 🧠「大模型」2小时完全从0训练64M的小参数LLM！Train a 64M-parameter LLM from scratch in just 2h!

MLxDL. Jupyter Notebook

open-r1. Fully open reproduction of DeepSeek-R1

nanoGPT. The simplest, fastest repository for training/finetuning medium-sized GPTs.

picoGPT. Python

aider. aider is AI pair programming in your terminal

Alpha-SQL. 🔥[ICML'25] Official repository for the paper "Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search"

ChatBase. Python

DB-GPT. An LLM Based Diagnosis System (https://arxiv.org/pdf/2312.01454.pdf)

ReFoRCE. A Text-to-SQL Agent with Self-Refinement, Format Restriction, and Column Exploration

Spider2. [ICLR 2025 Oral] Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows

dataherald. Interact with your SQL database, Natural Language to SQL using LLMs

NL2SQL_Handbook. This is a continuously updated handbook for readers to easily track the latest Text-to-SQL techniques in the literature and provide practical guidance for researchers and practitioners.

Chat2DB. AI-driven database tool and SQL client, The hottest GUI client, supporting MySQL, Oracle, PostgreSQL, DB2, SQL Server, DB2, SQLite, H2, ClickHouse, and more.

deep-learning-from-scratch-4. 『ゼロから作る Deep Learning ❹』(O'Reilly Japan, 2022)

Hands-on-RL. https://hrl.boyuai.com/

SWE-agent. SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024]

spring-ai-examples. Java

llm-apps-java-spring-ai. Samples showing how to build Java applications powered by Generative AI and LLMs using Spring AI and Spring Boot.

dify. Production-ready platform for agentic workflow development.

dify-plugin-tools-mcp_sse. Dify 1.0 Plugin MCP HTTP with SSE or Streamable HTTP transport Tools

Roo-Code. Roo Code gives you a whole dev team of AI agents in your code editor.

mcp-dbutils. 数读 是一件可以让你的大模型安全连接到数据库的MCP工具。| DButils is an all-in-one MCP service that enables your AI to do data analysis by harnessing versatile types of database (sqlite, mysql, postgres, and more) within a unified configuration of multiple connections in a secured way (like SSL and controlled write access).

mcp-client-cli. A simple CLI to run LLM prompt and implement MCP client.

crewAI. Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.

Geek02. 极客时间《DeepSeek应用开发实战课》配套代码

semantic-router. Superfast AI decision making and intelligent processing of multi-modal data.

ragflow. RAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context layer for LLMs

DeepSeek-V3. Python

els-cas-templates. Elsevier template 'els-cas-templates'.

System_Architect. 🐍 2024年软考 (杭州e类+租房补贴)系统架构设计师资料整理复习(软考高级) 二战 20天备考通过🎉🎉🎉2024年5月份的上半年系统架构师(压线过 52/46/45) 语雀文章总结体验 https://www.yuque.com/hakusai/el7lwd

BMCourse. The repo for Tsinghua summer course: Interdisciplinary Seminar on Big Models

db-readings. Readings in Databases

datahub. The Context Platform for your Data and AI Stack

louvain. Java implementation of the Louvain method of community detection in graphs

Awesome-Multimodal-Large-Language-Models. :sparkles::sparkles:Latest Advances on Multimodal Large Language Models

EFAK. A AI-Driven, Distributed and high-performance monitoring system, for comprehensive monitoring and management of kafka cluster.

SWIM. Statistical Workload Injector for MapReduce - Project at UC Berkeley AMP Lab

sofa-jraft. A production-grade java implementation of RAFT consensus algorithm.

rocksdb. A library that provides an embeddable, persistent key-value store for fast storage.

nebula-dev-docker. Nebula docker image for development

nebula-gears. Gears for Nebula Graph

nebula-third-party. Project to build third-party dependencies of Nebula Graph

gluten. Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.

chatgpt. ChatGPT网址导航，分享免费好用AI网站！

CloudEon. CloudEon uses Kubernetes to install and deploy open-source big data components, enabling the containerized operation of an open-source big data platform. This allows you to reduce your focus on underlying resource management and maintenance.

LeSystemDetection. 这是一款适用于Ubuntu(基于12.04)系统的操作系统测评软件，名为乐测评， 主要对系统进行如下六个方面的性能检测：CPU、内存、磁盘&IO、网络、开机速度、系统安全，软硬件方面都会进行较全面的检测。每个方面先检测操作系统基本信息，让用户对自己的系统有个基本了解。然后进行比较专业的性能测试，网络方面还有安全测试。总体上我们的功能很全，系统性能测试是主体，还有很多系统基本信息，给用户提供了比较全面的服务。

system_architect. 💯2026年 系统架构设计师（软考高级）备考资源库+配套免费刷题软件。

hadoop-2.6.0-cdh5.14.2. apache

calcite-test. Test code for apache calcite

HiBench. HiBench is a big data benchmark suite.

flink-cdc. Flink CDC is a streaming data integration tool

flink-connector-clickhouse. Flink SQL connector for ClickHouse. Support ClickHouseCatalog and read/write primary data, maps, arrays to clickhouse.

MatlabFunc. Matlab codes for feature learning

hashingSearch. Search with a hash index

bcddb. cross-architecture binary comparison database

BigDataAnalysis_Exp4. 实时大数据分析_文本相似-Shingling、Minhash算法

spark-neighbors. Spark-based approximate nearest neighbor search using locality-sensitive hashing

Spark. Scala

ldbc_snb_datagen_hadoop. The Hadoop-based variant of the SNB Datagen

WeDataSphere. WeDataSphere is a financial grade, one-stop big data platform suite.

nebula-bench. Collection of benchmark services and tools for Nebula Graph

nebula. A distributed, fast open-source graph database featuring horizontal scalability and high availability

resa-drs. Dynamic Resource Scheduling of RESA

papers-notebook. :page_facing_up: :cn: :page_with_curl: 论文阅读笔记（分布式系统、虚拟化、机器学习）Papers Notebook (Distributed System, Virtualization, Machine Learning)

PAS. Performance-Aware Job Scheduling

aqua_rowhammer_mitigation. AQUA (IEEE MICRO 2022) Artifact

gem5_bulkbitwise_PIM_consistency. C++

snailtrail. SnailTrail implementation

Ceph_Kariz_Spark. Students project of getting DAG out of Spark

ds2. DS2 is an auto-scaling controller for distributed streaming dataflows

robinhoodcache. RobinHood the first tail latency-aware caching system

panorama. Panorama: Capturing and Enhancing In Situ System Observability for Failure Detection

pLUTo. pLUTo is a DRAM-based Processing-using-Memory architecture that leverages the high density of DRAM to enable the massively parallel storing and querying of lookup tables (LUTs)

Hermes. A speculative mechanism to accelerate long-latency off-chip load requests by removing on-chip cache access latency from their critical path, as described by MICRO 2022 paper by Bera et al. (https://arxiv.org/pdf/2209.00188.pdf)

AiIndex. ChatGPT

NuRaft. C++ implementation of Raft core logic as a replication library

Graph-Cut. Graph-cut Image Segmentation ---------------------------- Implements Boykov/Kolmogorov’s Max-Flow/Min-Cut algorithm for computer vision problems. Two gray-scale images have been used to test the system for image segmentation (foreground/background segmentation) problem. Steps: 1. defined the graph structure and unary and pairwise terms. For graph structure, i have used available packages/libraries such as PyMaxflow. 2. likelihood function for background and foreground has been generated. 3. General energy function consisting of unary and pairwise energy functionals have been written. 4. Likelihood maps (intensity map ranging from 0 to 1) for foreground and background have been displayed. 5. Use Boykov/Kolmogorov maxflow / mincut approach for solving the energy minimization problem. 6. Final segmentation have been displayed. Created an image for which the background pixels are red, and the foreground pixels have the color of the input image. Relevant paper can be found here: http://www.csd.uwo.ca/~yuri/Papers/pami04.pdf

Graph_Cut. 2019-2020上 BIT 数据图像处理大作业-图割

awesome-ChatGPT-resource-zh. 精选 OpenAI 的 [ChatGPT](https://chat.openai.com) 资源清单, 跟随最新资源并添加中文相关Work

emqx. The most scalable and reliable MQTT broker for AI, IoT, IIoT and connected vehicles

ceph. Ceph is a distributed object, block, and file storage platform

juicefs. JuiceFS is a distributed POSIX file system built on top of Redis and S3.

cubefs. cloud-native distributed storage

velox. A composable and fully extensible C++ execution engine library for data management systems.

cmpp.

Python-Median-Filter. Simple implementation of median filter in python to remove noise from the images.

FGVC-PIM. Pytorch implementation for "A Novel Plug-in Module for Fine-Grained Visual Classification". fine-grained visual classification task.

Intrusion-Detection-Systems. This is the repo of the research paper, "Evaluating Shallow and Deep Neural Networks for Network Intrusion Detection Systems in Cyber Security".

network-traffic-predict. use lstm&svr to predict the network traffic

BaiduTraffic. This repo includes introduction, code and dataset of our paper Deep Sequence Learning with Auxiliary Information for Traffic Prediction (KDD 2018).

traffic_prediction. Traffic prediction is the task of predicting future traffic measurements (e.g. volume, speed, etc.) in a road network (graph), using historical data (timeseries).

sysu-thesis. 中山大学 LaTeX 毕业论文项目模板

Awesome-Fine-grained-Visual-Classification. Awesome Fine-grained Visual Classification

Image-document-extract-and-correction. 数字图像课程大作业，实现图片中文档提取与矫正。整体思路是通过hough变换检测出直线，进而得到角点，最后经过投影变换，进行矫正。整个项目只用到了opencv的IO操作(包括手写卷积，hough哈夫变换，投影变换等等)

deskew. Detect image skew angle and deskew image

awesome-image-classification. A curated list of deep learning image classification papers and codes

FER-VT. The unofficial implementation of paper "Facial expression recognition with grid-wise attention and visual transformer"

PDSN. PDSN code for ICCV'2019 paper 《Occlusion Robust Face Recognition based on Mask Learning with Pairwise Differential Siamese Network》

SummaryOfLoanSuspension. 全国各省市停贷通知汇总

Programmer-Learning-materials. 程序员 学习资料，以及一些书籍电子书，如果对您有所帮助，还希望大家多多star，更多资料请关注《Java有货》微信公众号获取

the_silver_searcher. A code-searching tool similar to ack, but faster.

cpp_etudes. smart tools for source code study : cpptree.pl, calltree.pl, javatree.pl, java_calltree.pl

neo4j. Graphs for Everyone

CS-Notes. :books: 技术面试必备基础知识、Leetcode、计算机操作系统、计算机网络、系统设计

sedona. A cluster computing framework for processing large-scale geospatial data

SparkInternals. Notes talking about the design and implementation of Apache Spark

hbase. Apache HBase

interview-questions. Interview Questions for Google, Amazon, Apple, etc. 根据超过 2000 篇真实面经整理的腾讯，阿里，字节跳动，Shopee，美团，滴滴高频面试题

eagle. Mirror of Apache Eagle

elasticsearch-sql. Use SQL to query Elasticsearch

elasticsearch-http-basic-auth-plugin. A simple and fast plugin adding HTTP basic authentication to Elasticsearch

elasticsearch-sql. parse sql into elasticsearch dsl with antlr4

ambari. Apache Ambari simplifies provisioning, managing, and monitoring of Apache Hadoop clusters.

BigData_AutomaticDeploy. 大数据自动化部署，包括自动化部署hadoop、hive、hbase、spark、storm等等一系列组件

ANTLR4-SqlBase. 剥离的模块，用于查看Spark SQL生成的语法树

vue-admin-template. a vue2.0 minimal admin template

vue-element-admin. :tada: A magical vue admin https://panjiachen.github.io/vue-element-admin

echarts. Apache ECharts is a powerful, interactive charting and data visualization library for browser

elasticsearch-definitive-guide. The Definitive Guide to Elasticsearch

elasticsearch. Free and Open Source, Distributed, RESTful Search Engine

winutils. winutils.exe hadoop.dll and hdfs.dll binaries for hadoop windows

flink-book-zh. Flink Tutorial Project

bahir-flink. Mirror of Apache Bahir Flink

BigData-Interview. :dart: :star2:[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结

paggan. Learning Face Age Progression: A Pyramid Architecture of GANs

BigDataGuide. 大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料

JavaBigData. 【大数据必备】非科班转行Java大数据面经分享

Flink-Kudu. Java library to integrate Flink and Kudu

Spark_ALS. 基于spark-ml,spark-mllib,spark-streaming的推荐算法实现

recsys_core. [推荐系统] Based on the scoring data set, the recommendation system is built with FM and LR as the core(基于评分数据集，构建以FM和LR为核心的推荐系统).

machine-learning-yearning-cn. Machine Learning Yearning 中文版 - 《机器学习训练秘籍》 - Andrew Ng 著

Spark-The-Definitive-Guide. Spark: The Definitive Guide's Code Repository

movie_recommender. MovieLens based recommender system.使用MovieLens数据集训练的电影推荐系统。

aliyun-log-flume. alibaba cloud log service flume sdk

common-regex. :jack_o_lantern: 常用正则表达式 - 收集一些在平时项目开发中经常用到的正则表达式。

tensorflow_poems. 中文古诗自动作诗机器人，x炸天，基于tensorflow1.10 api，正在积极维护升级中，快star，保持更新！

HarmonyOS. A curated list of awesome things related to HarmonyOS. 华为鸿蒙操作系统。

nacos. an easy-to-use dynamic service discovery, configuration and service management platform for building AI cloud native applications.

stargan. StarGAN - Official PyTorch Implementation (CVPR 2018)

showdoc. ShowDoc is a tool greatly applicable for an IT team to share documents online一个非常适合IT团队的在线API文档、技术文档工具

apollo. Apollo is a reliable configuration management system suitable for microservice configuration management scenarios.

TDengine. High-performance, scalable time-series database designed for Industrial IoT (IIoT) scenarios

SPADE. Semantic Image Synthesis with SPADE

d2l-zh. 《动手学深度学习》：面向中文读者、能运行、可讨论。中英文版被70多个国家的500多所大学用于教学。

Spoken_Number_Recognition. Speech recognition demo (Tensorflow + Keras + CNN) for spoken numbers.

keras. Deep Learning for humans

python-machine-learning-book. The "Python Machine Learning (1st edition)" book code repository and info resource

sqlflow. Brings SQL and AI together.

elasticsearch-definitive-guide-cn. Elasticsearch权威指南中文版

elasticsearch-definitive-guide. 欢迎加QQ群：109764489，贡献力量！

spring-boot-examples. about learning Spring Boot via examples. Spring Boot 教程、技术栈示例代码，快速简单上手教程。

star-history. The de facto GitHub star history graph.

flink. Apache Flink

CoolplaySpark. 酷玩 Spark: Spark 源代码解析、Spark 类库等

disco-java. Service discovery through Zookeeper for Java

kazoo. Kazoo is a high-level Python library that makes it easier to use Apache Zookeeper.

java-sdk. 百度AI开放平台 Java SDK

terminal. The new Windows Terminal and the original Windows console host, all in the same place!

javacv. Java interface to OpenCV, FFmpeg, and more

support.996.ICU. Microsoft and GitHub Workers Support 996.ICU

Coursera-ML-AndrewNg-Notes. 吴恩达老师的机器学习课程个人笔记

httpclientutil. 该项目基于HttpClient-4.4.1封装的一个工具类。提供了更为简单的GET、POST及上传下载等功能。同时支持插件式配置Http-Header（包含自定义header）、配置SSL和Proxy等。

Machine-Learning-with-Spark-Second-Edition. Machine Learning with Spark - Second Edition, by Packt

spark-notes. Deep Dive into Apache Spark 深入研读Spark源码

996.ICU. Repo for counting stars and contributing. Press F to pay respect to glorious developers.

incubator-seata. :fire: Seata is an easy-to-use, high-performance, open source distributed transaction solution.

NewsRecommendSystem. 个性化新闻推荐系统，A news recommendation system involving collaborative filtering,content-based recommendation and hot news recommendation, can be adapted easily to be put into use in other circumstances.

deeplearningbook-chinese. Deep Learning Book Chinese Translation

x-deeplearning. An industrial deep learning framework for high-dimension sparse data

EmojiPackage. 表情包资源合集，张张都是经典

AndroidMaterialDesignUI. Material_Design_UI

raft-zh_cn. Raft一致性算法论文的中文翻译

raft-java. Raft Java implementation which is simple and easy to understand.

kudu. Mirror of Apache Kudu

HanLP. 中文分词 词性标注 命名实体识别 依存句法分析 成分句法分析 语义依存分析 语义角色标注 指代消解 风格转换 语义相似度 新词发现 关键词短语提取 自动摘要 文本分类聚类 拼音简繁转换 自然语言处理

hibeaver. HiBeaver is a gradle plugin for java byte code manipulation and AOP design by modifying project byte code during build of the package, or modifying byte code within Jar independently.

Logan. Logan is a lightweight case logging system based on mobile platform.

presto. The official home of the Presto distributed SQL query engine for big data

gitbook. The open source frontend for GitBook doc sites

spark. Apache Spark - A unified analytics engine for large-scale data processing