Lijie Xu

SparkInternals. Notes talking about the design and implementation of Apache Spark

SparkLearning. Learning to write Spark examples

ApacheSparkBook. Scala

MyNotes. Self-written notes that may be useful

blogs. My blogs

SparkProfiler. Profiling Spark Applications for Performance Comparison and Diagnosis

SparkFaultBench. A Spark Reliability Testing Suite

MySlides. Self-written slides that may be useful

AgentMemorySystem. Python

SparkGC. Scala

CorgiPile-PyTorch. The implementation of CorgiPile in PyTorch

my.github.io. My Homepage

SparkBench. Spark benchmark

CorgiPile-PostgreSQL. Code of "In-Database Machine Learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle"

bagua. Bagua Speeds up PyTorch

streaming-benchmarks. Benchmarks for Low Latency (Streaming) solutions including Apache Storm, Apache Spark, Apache Flink, ...

DistStream. Scala

SparkExamples. Basic examples for learning Spark

Table-SGD. Python

bandwidth-bench. Measure memory and disk bandwidth using the random access size as a paramater.

BenchmarkScripts. Benchmark scripts in master

enhanced-Eclipse-MAT. extracting framework & user objects from task's heap dump

SparkStreamingBench. Testing SparkStreaming

Misc. Store miscellaneou things

Spark-core. The core library from Spark

MyPaper. My papers and technique reports

spark-sql-perf. Scala

TableRFL. A Framework for Relational Federated Learning

JerryLead.github.io. My homepage

Text-Classification. PyTorch implementation of some text classification models (HAN, fastText, BiLSTM-Attention, TextCNN, Transformer) | 文本分类

FMEM. A Fine-grained Memory Estimator for MapReduce Jobs

MemoryEstimator. Python

madlib. Mirror of Apache MADlib

TR. My technical reports

moa. MOA is an open source framework for Big Data stream mining. It includes a collection of machine learning algorithms (classification, regression, clustering, outlier detection, concept drift detection and recommender systems) and tools for evaluation.

HadoopJobInfoCollector. Fetch the configuration/timeline/counters/log infos from JobTracker

GraphExamples. Examples of GraphX

AndroidGCProfiler. Profiling the GC activities in Android ART JVM

BigDataBenchmark. Representative MapReduce Job for Hadoop Benchmark

PyTorch-LBFGS. A PyTorch implementation of L-BFGS.

TablePuppet. Python

spark. Mirror of Apache Spark

pytorch-minimize. Newton and Quasi-Newton optimization with PyTorch