yifan @yifan0sun

I like optimization, learning, and anything algorithmic-y. I'm also easily distracted by completely irrelevant topics. Joined May 2021

Tweets

109
Followers

166
Following

184
Likes

216

yifan @yifan0sun

4 months ago

What happens when we have access to two different explainability metrics: gradients *and* attention weights on transformers? How do they differ per layer? How are they affected by task complexity, or model choice? Click here to explore more: yifan0sun.github.io/BERTGradGraph/ See more at…

0 0 0 80 0

Download Image

yifan @yifan0sun

4 months ago

How does task complexity and optimality really affect loss sharpness/flatness? What does sharpness really look like, especially along the training path? Check out this loss landscape visualizer! yifan0sun.github.io/LossLandscapeV… See more at optimalvisualizer.com

0 1 2 212 3

Download Image

yifan @yifan0sun

4 months ago

There are a lot of intuitions and studies as to which layers are responsible for what complexity of task, and what fine-tuning does to the saliency of attention layers. Now you can visualize it in real-time! bert-attention-visualizer.vercel.app See more at optimalvisualizer.com

0 0 0 72 0

Download Image

yifan @yifan0sun

8 months ago

Ok can someone tell me what tools people are using to make these amazing simulation videos? I feel so technologically behind. What people are doing today seems so professional, and the sheer volume suggests a tool out there that does it much faster than whatever I'm using

0 0 1 163 0

yifan @yifan0sun

10 months ago

Blog post: what does it mean to be subgaussian? thehappyoptimist.com/2024/10/29/wha… Tl;dr: shouldn't the subgaussian bound be parameter-dependent? Somehow just basing it on the size of the support seems too loose. am I overthinking it?

0 0 3 168 1

Xueying Bai @Bai1994Xueying

11 months ago

We will present our #COLM2024 paper, Does RoBERTa Perform Better than BERT in Continual Learning: An Attention Sink Perspective, on Monday 11:00 AM – 1:00 PM at #20 Poster Area. Please stop by if you are interested! Paper: openreview.net/pdf?id=VHhwhmt… Code: github.com/StonyBrookNLP/…

1 7 9 821 1

Download Image

yifan @yifan0sun

a year ago

What are people's favorite tablet platform?When I use GoodNote, I can't seem to fit a lot on the page, but when I use jamboard equivalents, the technology just isn't as good. Is there something that mimics a blackboard better, where you can fit a lot of content on one page?

2 0 0 880 0

yifan @yifan0sun

a year ago

Combining electrical engineering with sparse matrices! Old news, revisited. thehappyoptimist.com/2024/04/16/spe…

0 0 1 199 1

yifan @yifan0sun

3 years ago

Question for twitterverse: how hard is it to implement XGboost from scratch (or some barebones version of it?) I'm thinking of adding it as an undergrad assignment. Would that be cruel or just eccentric?

3 0 1 462 0

yifan @yifan0sun

3 years ago

Whew, disappeared for a few months to do some teaching. Updated my giant pile of machine learning problems, which I've used in the undergrad and graduate course. Do let me know if you find them useful! sites.google.com/site/yifansunw…