Recent Research Projects

NeuGraph (NGra): System for Graph Neural Networks (GNNs)

Recent deep learning models have moved beyond low-dimensional regular grids to high-dimensional graph-structured data, leading to large graph-based irregular and sparse models that go beyond what existing DL frameworks are designed for. We introduce NeuGraph (NGra), a parallel processing framework for graph-based deep neural networks (GNNs) on top of existing DL frameworks.

  • NeuGraph presents a new programming model for expressing deep neural networks as vertex programs. This model not only allows GNNs to be expressed intuitively, but also facilitates the mapping to an efficient dataflow representation.
  • NeuGraph addresses the scalability challenge transparently through automatic graph partitioning and chunk-based stream processing out of GPU core or over multiple GPUs.
  • NeuGraph achieves efficiency through highly optimized Scatter/Gather operations on GPU despite graph sparsity.

Garaph: GPU-accelerated Graph Processing System

Recent advances in storage and accelerators provide the opportunity to efficiently process large-scale graphs on a single machine. Thus, we present Garaph, a GPU-accelerated graph processing system. Garaph is novel in three ways:

  • First, Garaph proposes a vertex replication degree customization scheme that maximizes the GPU utilization given vertices’ degrees and space constraints.
  • Second, Garaph adopts a balanced edge-based partition method, ensuring sequential memory access and load balance over CPU threads, and also a hybrid of notify-pull and pull computation models optimized for fast graph processing on the CPU.
  • Third, Garaph designs a workload scheduler which takes into account both characteristics of processing elements and graph algorithms.

Publications

Selected Publications (The full publication list is here.)

Welder: Scheduling Deep Learning Memory Access via Tile-graph
Yining Shi, Zhi Yang, Jilong Xue, Lingxiao Ma, Yuqing Xia, Ziming Miao, Yuxiao Guo, Fan Yang, Lidong Zhou
The 17th USENIX Symposium on Operating Systems Design and Implementation (OSDI '23), 2023

Rammer: Enabling Holistic Deep Learning Compiler Optimizations with rTasks
Lingxiao Ma, Zhiqiang Xie, Zhi Yang, Jilong Xue, Youshan Miao, Wei Cui , Wenxiang Hu, Fan Yang, Lintao Zhang, Lidong Zhou
the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI '20), 2020

HiveD: Sharing a GPU Cluster for Deep Learning with Guarantees
Hanyu Zhao, Zhenhua Han, Zhi Yang, Quanlu Zhang, Fan Yang, Lidong Zhou, Mao Yang, Francis C.M. Lau, Yuqi Wang, Yifan Xiong, Bin Wang
the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI '20), 2020

HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework
Xupeng Miao, Hailin Zhang, Yining Shi, Xiaonan Nie, Zhi Yang, Yangyu Tao, Bin Cui
International Conference on Very Large Data Bases (VLDB), 2022, Best Scalable Data Science Paper

PASCA: a New Paradigm and System to Build Scalable Graph Neural Network
Wentao Zhang, Yu Shen, Zheyu Lin, Yang Li, Xiaosen Li, wen ouyang, Yangyu Tao, Zhi Yang, Bin Cui
The Web Conference (WWW), 2022, Best Student Paper Award

NeuGraph: Parallel Deep Neural Network Computation on Large Graphs
Lingxiao Ma, Zhi Yang, Youshan Miao, Jilong Xue, Ming Wu, Lidong Zhou, Yafei Dai
USENIX Annual Technical Conference (USENIX ATC), 2019

Garaph: Efficient GPU-accelerated Graph Processing on a Single Machine with Balanced Replication
Lingxiao Ma, Zhi Yang, Han Chen, Jilong Xue, Yafei Dai
USENIX Annual Technical Conference (USENIX ATC), 2017

SiloD: A Co-design of Caching and Scheduling for Deep Learning Clusters
Hanyu Zhao, Zhenhua Han, Zhi Yang, Quanlu Zhang, Mingxia Li, Fan Yang, Qianxi Zhang, Binyang Li, Yuqing Yang, Lili Qiu, Lintao Zhang, Lidong Zhou
The European Conference on Computer Systems (EuroSys), 2023

ALG: Fast and Accurate Active Learning Framework for Graph Convolutional Networks
Wentao Zhang, Yu Shen, Yang Li, Lei Chen, Zhi Yang, Bin Cui
International Conference on Management of Data (SIGMOD), 2021

Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce
Xupeng Miao, Xiaonan Nie, Yingxia Shao, Zhi Yang, Jiawei Jiang, Lingxiao Ma, Bin Cui
International Conference on Management of Data (SIGMOD), 2021

Grain: Improving Data Efficiency of Graph Neural Networks via Diversified Influence Maximization
Wentao Zhang, Zhi Yang, YeXin Wang, Yu Shen, Yang Li , Liang Wang, Bin Cui
International Conference on Very Large Data Bases (VLDB), 2021

VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition
Yang Li, Yu Shen, Wentao Zhang, Jiawei Jiang, Bolin Ding, Yaliang Li, Jingren Zhou, Zhi Yang, Wentao Wu, Ce Zhang and Bin Cui
International Conference on Very Large Data Bases (VLDB), 2021

RIM: Reliable Influence-based Active Learning on Graphs
Wentao Zhang, Yexin Wang, Zhenbang You, Meng Cao, Ping Huang, Jiulong Shan, Zhi Yang, Bin Cui
Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS), 2021, Spotlight.

Node Dependent Local Smoothing for Scalable Graph Learning
Wentao Zhang, Mingyu Yang, Zeang Sheng, Yang Li, Wen Ouyang, Yangyu Tao, Zhi Yang, Bin Cui
Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS), 2021, Spotlight.

SDPaxos: Building Efficient Semi-Decentralized Geo-replicated State Machines
Hanyu Zhao, Quanlu Zhang, Zhi Yang, Ming Wu and Yafei Dai
ACM Symposium on Cloud Computing 2018 (SoCC), 2018

VoteTrust: Leveraging Friend Invitation Graph to Defend against Social Network Sybils
Zhi Yang, Jilong Xue, Xiaoyong Yang, Xiao Wang, Yafei Dai
IEEE Transactions on Dependable and Secure Computing (TDSC), 2015

A game theoretic model for the formation of navigable small-world networks
Zhi Yang and Wei Chen
International World Wide Web Conference (WWW), 2015

CHARM: A Cost-efficient Multi-cloud Data Hosting Scheme with High Availability
Quanlu Zhang, Shenglong Li, Zhenhua Li, Yuanjian Xing, Zhi Yang, and Yafei Dai
IEEE Transaction on Cloud Computing (TCC), 2015, (Spotlight Paper)

Uncovering Social Network Sybils in the Wild
Zhi Yang , Christo Wilson, Xiao Wang, Tingting Gao, Ben Y. Zhao and Yafei Dai
ACM Transactions on Knowledge Discovery from Data (TKDD), 2014, (citation：505)

Services

PC member

The Web Conference (WWW) 2021
International Conference Series on Advances in Social Network Analysis and Mining (ASONAM) 2020
ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) 2020
The Web Conference (WWW) 2020
Annual Conference on Database Systems for Advanced Applications (DASFAA) 2020
International Joint Conference on Artificial Intelligence (IJCAI) 2020
ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) 2019
International Conference Series on Advances in Social Network Analysis and Mining (ASONAM) 2019
IEEE International Conference on Big Data (IEEE BigData) 2019
IEEE International Conference on Big Data (IEEE BigData) 2018
The Web Conference (WWW) 2017 (Poster Track)