Junyi (Jenny) Wei
I am currently a PhD candidate at Math Department in University of Wisconsin, Madison, supervised by Prof. Yingyu Liang. Previously, I received bachelor's Degree at Hong Kong University of Science and Technology where I was advised by Prof. K. Y. Michael Wong.
I am actively looking for Industry Research and Engineer Positions, earliest start from Sep. 2024. Do not hesitate to drop me an email if you are interested
jwei53@wisc.edu
+1 (626) 764 1115'
LinkedIn
Github
Education
University of Wisconsin, Madison
Jul 2017 - Now | Ph. D. Candidate in Mathematics
Computer vision, large language models, computational math, advanced algorithm, project organization skills.
University of Wisconsin, Madison
Aug 2021 - Jun 2022 | Master of Science in Computer Science
OOP (Java and Python), basic algorithms, deep learning theory and experiments, probability, database (MySQL)
Hong Kong University of Science and Technology (HKUST)
Sep 2013 – July 2017 | Bachelor of Science in Pure Mathematics and Physics
Probability, linear algebra, differential equations, analysis, geometry, statistical mechanics, quantum mechanics
Swiss Federal Institute of Technology in Zürich (ETH Zürich)
Sep 2016 – Feb 2017 | Visiting Scholar
Skills
PyTorch, Deep Learning Programming & Analysis, Python, MATLAB, Julia, Java, C, MySQL, HTML, Latex
Selected Publications
Why Larger Language Models Do In-context Learning Differently?
ZhenmeiShi, JunyiWei, ZhuoyanXu, YingyuLiang
  • Give a theoretical and empirical explaination to the interesting phenomenon that larger language models are less robust comparing to smaller language models if given noisy prompt with flipped labels.
  • On going project. See our workshop version presented in NeurIPS 2023 Workshop R0-FoMo
Improving Foundation Models For Few-Shot Learning Via Multitask Finetuning
Zhuoyan Xu, Zhenmei Shi, Junyi Wei, Yin Li, Yingyu Liang
  • Explore the paradigm of finetuning a foundation model before adapting to a target task, using a set of related tasks in a data efficient way. We show both theoretically and empirically that with a diverse set of related tasks finetuning leads to reduced error in the target task.
  • On going project. See our workshop version presented in ICLR 2023 Workshop Me-FoMo.
Provable Guarantees for Neural Networks via Gradient Feature Learning
Zhenmei Shi*, Junyi Wei*, Yingyu Liang
  • Propose a unified analysis framework for two-layer networks trained by gradient descent. Demonstrate the effectiveness of our framework in several prototypical problems such as mixtures of Gaussians and parity functions.
  • Accepted by NeurIPS 2023
A Theoretical Analysis on Feature Learning in Neural Networks: Emergence from Inputs and Advantage Over Fixed Features
Zhenmei Shi*, Junyi Wei*, Yingyu Liang
  • Theoretically prove the feature learning ability of neural networks observed in empirical works, and give convergence guarantees. Explain the advantages of neural networks over fixed feature methods like NTK.
  • Accepted by ICLR 2022
Commodity Price Modeling using Multi-Agent Network
Bin Li, K Y Michael Wong, Amos H M Chan, Tsz Yan So, Hermanni Heimonen, Junyi Wei and David Saad
  • Developed commodity price model by building up a Multi-Agent Network and numerically computing the Nash equilibrium solution. Accurately estimate the dynamics for different kinds of commodity prices. Provide reliable techniques for future market prediction.
  • Accepted by Journal of Statistical Mechanics: Theory and Experiment.
Selected Projects
Learning Region Representation Via Self-Supervised Learning of Temporal Correspondence
Junyi Wei*, Jitian Zhao*, Yibing Wei*
Designed a transformer-based self-supervised method to learn region representation in video segmentation task. The method efficiently learns the segmentation features using temporal correspondence naturally imbedded in video data, with no label required.
Adopting Linear Model to Accelerate Neural Network Training
Yin Liu, Junyi Wei, Zijun Ma, Hanying Jiang, Yihan Zhang
Using kernel machine to accelerate the neural network training in the first stage, and fine tune the model using stochastic gradient descent (SGD) in the second stage. Achieved the same accuracy with half training time comparing to vanilla SGD.
Experience
IFDS Research Assistant
May 2022 - May 2023 | University of Wisconsin-Madison
Teaching Assistant
Sep 2017 - May 2022, May 2023 - Now | University of Wisconsin-Madison
Researcher
May 2016 - Aug 2016 | École Normale Supérieure
Languages
English
Work proficiency
Mandarin Chinese
Native