it's a hierarchical navigable small world after all

Man goes to doctor. Says he feels all alone in a world not on track to solve alignment. Doctor says, 'Treatment is simple. Great alignment researcher Pagliacci is in town. Go and see him. He has a plan to solve alignment.' Man bursts into tears. 'But doctor.. I am Pagliacci'

concreteness precedes abstraction

the variance will increase until morale improves

startup idea: coil whine simulator so you can hear your cloud GPUs go brrr

the same procedure as every year

any swe can write code that's maintainable, but it takes a research engineer to write code that's barely maintainable

pretraining leakage disanalogy explained: we want to study the analogy where weak models supervise the strong model. but because our models are pretrained on human text, there's implicit supervision by something stronger. this could make results look better than they actually are

human simulator / imitation saliency problem explained: one very natural generalization is to just say ~what a human would say. if this is more natural than what the human would say if they knew what the AI knew, then it will systematically hide things humans can't understand

There's a cute formula that appears in this paper: KL[best-of-n||best-of-1] = log(n) - (n-1)/n, where best-of-n is the distribution of the best of n i.i.d. samples according to some scoring function. Several people have asked about this so I put together an explainer. (1/6)

I, for one, consider myself an optimist