Hyunjong Kim

ms-swift. Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

Awesome-Personalization-in-MLLMs. Awesome papers on personalization in LLMs/MLLMs: memory, alignment, retrieval, and evaluation.

VisualMem. Official PyTorch implementation of "Personal Visual Memory from Explicit and Implicit Evidence"

PersonaVLM. [CVPR 2026 Highlight] PersonaVLM: Long-Term Personalized Multimodal LLMs

Persona-DB. Python

CultureBank. Python

VITAL. Code of our paper - "VITAL: A New Dataset for Benchmarking Pluralistic Alignment in Healthcare" (Accepted to ACL 2025 (Main Proceedings))

Agent-ValueBench. Python

MoralCoT. Repo for: When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment

delphi. Python

ethics. Aligning AI With Shared Human Values (ICLR 2021)

hh-rlhf. Human preference data for "Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback"

moral_stories. Data and code for the "Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences" (Emelin et al., 2021) paper.

moralchoice. Evaluating the Moral Beliefs Encoded in LLMs

scruples. A corpus and code for understanding norms and subjectivity. 🤖

ValueBench. [ACL 2024] ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models

kaleido. Python

OpenCharacterTraining. Open Character Training

EigenBench. Python

daily_dilemmas. Python

SPARTA_WSDM2022. This repository contains the code and dataset for our paper titled Speaker and Time-aware Joint Contextual Learning for Dialogue-act Classification in Counselling Conversations accepted at WSDM Conference, 2022.

VLM-R1. Solve Visual Understanding with Reinforced VLMs

verl. verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

verl-internvl. Python

trl. Train transformer language models with reinforcement learning.

InternVL. [CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型

text-image-relationship. Text-Image Relationships (ACL 2019)

Chimera. The official codes for our paper "Exploring Cognitive and Aesthetic Sentiment Causality for Multimodal Aspect-based Sentiment Classification"

JML. Python

LLM4Annotation.

ekphrasis. Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).

MulCoT-RD. Python

GMP. Generated Multimodal Prompt

Llama3.2-Vision-Finetune. An open-source implementaion for fine-tuning Llama3.2-Vision series by Meta.

Gemma3-Finetune. An open-source implementaion for Gemma3 series by Google.

LlamaFactory. Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

MDCA. This repo provides the source code and data for the paper Dual Causes Generation Assisted Model for Multimodal Aspect-based Sentiment Classification. (TNNLS 2024)

PlugLLM-MABSA. Python

MCPL. The source code for MCPL.

CORSA. Paper Multimodal Aspect-Based Sentiment Analysis under Conditional Relation(COLING 2025)'s code and data

AoM. Codes for AoM: Detecting Aspect-oriented Information for Multimodal Aspect-Based Sentiment Analysis

VLP-MABSA. Python

KEF. [COLING 2022] Learning from Adjective-Noun Pairs: A Knowledge-enhanced Framework for Target-Oriented Multimodal Sentiment Classification

CMMT-Code. Python

DQPSA. Python

llm-annotations-annomatic. "The promises and pitfalls of LLM annotations: a case study on media bias detection" investigates the usability and generalization of classifiers trained with LLM annotators.

RethinkingTMSC. [EMNLP 2023] RethinkingTMSC: An Empirical Study for Target-Oriented Multimodal Sentiment Classification

Qwen3-VL. Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.

Qwen-VL-Series-Finetune. An open-source implementaion for fine-tuning Qwen-VL series by Alibaba Cloud.

Qwen-VL. The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.

ESAFN. Dataset and codes for our paper "Entity-Sensitive Attention and Fusion Network for Entity-Level Multimodal Sentiment Classification".

TomBERT. Dataset and codes for our IJCAI 2019 paper "Adapting BERT for Target-Oriented Multimodal Sentiment Classification"

artemis-v2. Code for the paper: It is Okay to Not Be Okay: Overcoming Emotional Bias in Affective Image Captioning by Contrastive Data Collection

Image-Emotion-Datasets. The datasets for image emotion computing

vaderSentiment. VADER Sentiment Analysis. VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment analysis tool that is specifically attuned to sentiments expressed in social media, and works well on texts from other domains.

EmoBank. This repository contains EmoBank, a large-scale text corpus manually annotated with emotion according to the psychological Valence-Arousal-Dominance scheme.

XANEW. Jupyter Notebook

siebert. Jupyter Notebook

arxiv-latex-cleaner. arXiv LaTeX Cleaner: Easily clean the LaTeX code of your paper to submit to arXiv

ShuttleNet. Official Implementation for ShuttleNet: Position-aware Fusion of Rally Progress and Player Styles for Stroke Forecasting in Badminton (AAAI'22)

multimodal-knowledge-graph. A collection of resources on multimodal knowledge graph, including datasets, papers and contests.

awesome-captioning-evaluation. [IJCAI 2025] Image Captioning Evaluation in the Age of Multimodal LLMs: Challenges and Future Perspectives

cs-paper-checklist. A final sanity checklist to help your CS paper get accepted, not desk rejected.

speculative-decoding. Explorations into some recent techniques surrounding speculative decoding

roformer. Rotary Transformer

streaming-llm. [ICLR 2024] Efficient Streaming Language Models with Attention Sinks

speculative-sampling. Simple implementation of Speculative Sampling in NumPy for GPT-2.

Spec-Bench. Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)

LongSpec. [ACL 2026 (Main)] LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification

MagicDec. [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding

TriForce. [COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding

EAGLE. Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).

SpeculativeDecodingPapers. 📰 Must-read papers and blogs on Speculative Decoding ⚡️

2027-AI-College-Jobs. 2027 AI/ML internship & new graduate job list updated daily

clair. CLAIR: A (surprisingly) simple semantic text metric with large language models.

InfoMetIC. Python

prometheus-vision. [ACL 2024 Findings & ICLR 2024 WS] An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on customized score rubric, Prometheus-Vision is a good alternative for human evaluation and GPT-4V evaluation.

SelfCorrectDecompose. Code for paper "Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMs"

LLaVA-CoT. [ICCV 2025] LLaVA-CoT, a visual language model capable of spontaneous, systematic reasoning

gveval. Official Repo for AAAI 2025 G-VEval: A Versatile Metric for Evaluating Image and Video Captions Using GPT-4o

pacscore. [CVPR 2023 & IJCV 2025] Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation

licenses. Licenses approved by Team Digitale, ready to use in your repository

DENEB. [ACCV2024] DENEB: A Hallucination-Robust Automatic Evaluation Metric for Image Captioning

LLaVA-1.6-ft. [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

LLaVA-NeXT-Image-Llama3-Lora. LLaVA-NeXT-Image-Llama3-Lora, Modified from https://github.com/arielnlee/LLaVA-1.6-ft

LLaVA-NeXT. Python

TransBERT-ijcai2019. Code used in our ijcai 2019 paper "Story Ending Prediction by Transferable BERT"

SentiWSP. Python

Polos. [CVPR24 Highlights] Polos: Multimodal Metric Learning from Human Feedback for Image Captioning

UMIC. An unreferenced image captioning metric (ACL-21)

scaling_sentemb. Scaling Sentence Embeddings with Large Language Models

unilm. Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

Sentence-Embedding-Is-All-You-Need. Korean Sentence Embedding Repository

acl-style-files. Official style files for papers submitted to venues of the Association for Computational Linguistics

CoreNLP. CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.

stanford-corenlp. Python wrapper for Stanford CoreNLP.

text-generation-inference. Large Language Model Text Generation Inference

llama3-langchain-kor. It shows a korean chatbot using LangChain based on Llama3

FLEUR. [ACL 2024] FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model

llama3. The official Meta Llama 3 GitHub site

mistral-inference. Official inference library for Mistral models

paramount. Agent accuracy measurements for LLMs

prometheus-eval. Evaluate your LLM's response with Prometheus and GPT4 💯

prometheus. [ICLR 2024 & NeurIPS 2023 WS] An Evaluator LM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on a customized score rubric, Prometheus is a good alternative for human evaluation and GPT-4 evaluation.

edc. Python

FActScore. A package to evaluate factuality of long-form generation. Original implementation of our EMNLP 2023 paper "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation"

HaluEval. This is the repository of HaluEval, a large-scale hallucination evaluation benchmark for Large Language Models.

semantic_uncertainty. Python

techqa. The TechQA dataset -- http://ibm.biz/Tech_QA

VKGDR. Python

mental_chatbot_survey. This is the github repository for the paper: An Integrative Survey on Mental Health Conversational Agents to Bridge Computer Science and Medical Perspectives

chat_templates. Chat Templates for 🤗 HuggingFace Large Language Models

AutoKG. [WWWJ 2024] LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities

KG-LLM-MDQA. Python

geval. Code for paper "G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment"

AutoKG. Efficient Automated Knowledge Graph Generation for Large Language Models

spert. PyTorch code for SpERT: Span-based Entity and Relation Transformer

trimf. Code for "A Trigger-Sense Memory Flow Framework for Joint Entity and Relation Extraction". accepted at WWW 2021.

PL-Marker. Source code for "Packed Levitated Marker for Entity and Relation Extraction"

CasRel. [ACL 2020] A Novel Cascade Binary Tagging Framework for Relational Triple Extraction

AmpliGraph. Python library for Representation Learning on Knowledge Graphs https://docs.ampligraph.org

pytorch_graph-rel. A PyTorch implementation of GraphRel

TPlinker-joint-extraction. Jupyter Notebook

DEER. The implementation for "DEER: Descriptive Knowledge Graph for Explaining Entity Relationships" (EMNLP '22)

Awesome-LLM-KG. Awesome papers about unifying LLMs and KGs

KG-LLM-Papers. [Paper List] Papers integrating knowledge graphs (KGs) and large language models (LLMs)

Template. TeX

expert_laypeople_reflection_annotation. Investigation of expert vs laypeople annotation of reflective statements in motivational counselling.

KoT5. 한국어 T5 모델

komt. Korean Multi-task Instruction Tuning

simpleT5. simpleT5 is built on top of PyTorch-lightning⚡️ and Transformers🤗 that lets you quickly train your T5 models.

t5x. Python

text-to-text-transfer-transformer. Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

therapist-observer. Code for the ACL 2019 paper "Observing Dialogue in Therapy: Categorizing and Forecasting Behavioral Codes"

direct-preference-optimization. Reference implementation for DPO (Direct Preference Optimization)

ontogpt. LLM-based ontological extraction tools, including SPIRES

long-ke-t5. Python

CR_RE_NER. Python

ke-t5-downstreams. Python

ke-t5. Python

KEMI.

flair. A very simple framework for state-of-the-art Natural Language Processing (NLP)

PromptRank. [ACL 2023] PromptRank: Unsupervised Keyphrase Extraction Using Prompt

DiffusionNER. Code for the paper "DiffusionNER: Boundary Diffusion for Named Entity Recognition", accepted at ACL 2023.

MRL-2021. English-Korean Parallel Dataset

Cognitive-Reframing. Codes and Datasets for our ACL 2023 paper on cognitive reframing of negative thoughts

doccano. Open source annotation tool for machine learning practitioners.

agents. An Open-source Framework for Data-centric, Self-evolving Autonomous Language Agents

conversational-uptake. Code and data for the paper "Measuring Conversational Uptake: A Case-Study on Student-Teacher Interactions"

Client-Reactions. The official repository of ACL2023 paper: Understanding Client Reactions in Online Mental Health Counseling

mathdial. 🧮 MathDial: A Dialog Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems, EMNLP Findings 2023

counsel-chat. This repository holds the code for working with data from counselchat.com

pdfdiff. Command-line tool to inspect the difference between (the text in) two PDF files

Boosting-with-MI-Strategy. Data and code repository for boosting distress support dialogue responses with Motivational Interviewing strategy

Instructdial. Code for the paper Code for the paper InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning

awesome-instruction-datasets. A collection of awesome-prompt-datasets, awesome-instruction-dataset, to train ChatLLM such as chatgpt 收录各种各样的指令数据集, 用于训练 ChatLLM 模型。

openai-python. The official Python library for the OpenAI API

KESA. The source code of KESA

SentiLARE. Codes for our paper "SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge" (EMNLP 2020)

Vicuna-LoRA-RLHF-PyTorch. A full pipeline to finetune Vicuna LLM with LoRA and RLHF on consumer hardware. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the Vicuna architecture. Basically ChatGPT but with Vicuna

Emotional-Support-Conversation. Data and codes for ACL 2021 paper: Towards Emotional Support Dialog Systems

Motivational-Interviewing-Dataset. This repository contains a dataset containing ≈2K dialogues whose listener utterances are annotated from labels derived from the Motiva- tional Interviewing Treatment Integrity (MITI) code.

AnnoMI. Official repository for the AnnoMI dataset: the first public collection of expert-annotated MI transcripts.

dialog-inpainting.

TextGAIL. Python

Few-Shot-Intent-Detection. Few-Shot-Intent-Detection includes popular challenging intent detection datasets with/without OOS queries and state-of-the-art baselines and results.

eswa-template. ESWA - Expert Systems with Applications latex template

awesome-human-in-the-loop. Awesome List of Human in the Loop resources and references for retraining models.

PMC-LLaMA. The official codes for "PMC-LLaMA: Towards Building Open-source Language Models for Medicine"

ChatDoctor. Python

align_uniform. Open source code for paper "Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere" ICML 2020

dygiepp. Span-based system for named entity, relation, and event extraction.

MatTPUSciBERT. Material SciBERT trained on TPU

SuperMat. Superconductors material dataset

Material_Synthesis_Corpus. Provides texts labeled with the unique expressions in the inorganic materials literature.

materials-synthesis-generative-models. Public release of data and code for materials synthesis generation

Awesome-LLM. Awesome-LLM: a curated list of Large Language Model

chemdataextractor2. ChemDataExtractor Version 2.0

chemdataextractor. Pipeline for automated extraction of chemical property information from scientific documents

SentiWordNet. The SentiWordNet sentiment lexicon

SentiBERT. Code for SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics (ACL'2020).

annotated-materials-syntheses.

sc-comics. Python

mat2vec. Supplementary Materials for Tshitoyan et al. "Unsupervised word embeddings capture latent knowledge from materials science literature", Nature (2019).

ColossalAI. Making large AI models cheaper, faster and more accessible

chatllama. ChatLLaMA 📢 Open source implementation for LLaMA-based ChatGPT runnable in a single GPU. 15x faster training process than ChatGPT

sofc-exp_textmining_resources. Resources related to ACL 2020 paper "The SOFC-Exp Corpus and Neural Approaches to Information Extraction in the Materials Science Domain"

science-parse. Science Parse parses scientific papers (in PDF form) and returns them in structured form.

Material_Science_Relation_Extraction. Python

MatBERT_NER. General training framework for NER models on MatBERT

RLHF. Collection of links, tutorials and best practices of how to collect the data and build end-to-end RLHF system to finetune Generative AI models

PaLM-rlhf-pytorch. Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

awesome-RLHF. A curated list of reinforcement learning with human feedback resources (continually updated)

grobid-superconductors. Grobid module for superconductor material and properties extraction

grobid-quantities. GROBID extension for identifying and normalizing physical quantities.

delft. a Deep Learning Framework for Text https://delft.readthedocs.io/

MatBERT. A pretrained BERT model on materials science literature

batterybert. BatteryBERT: A Pre-trained Language Model for Battery Database Enhancement

NLP4MatSci-ACL23. This repository contains the dataset and code for our ACL'23 publication: "MatSci-NLP: Evaluating Scientific Language Models on Materials Science Language Tasks Using Text-to-Schema Modeling"

data-resources-for-materials-science. A list of databases, datasets and books/handbooks where you can find materials properties for machine learning applications.