I've been working on a new product: MakeDistribution.com!

It's the best solution to creating a subjective probability distribution, i.e. one that reflects human judgement rather than being fit to data.

Sounds simple, but most existing solutions have important flaws (1/15)

Whoever made this graphic for ABC deserves a raise.

timeless

New: Open Medical LLM Leaderboard! 🩺

In basic chatbots, errors are annoyances.
In medical LLMs, errors can have life-threatening consequences 🩸

It's therefore vital to benchmark/follow advances in medical LLMs before thinking about deployment.

Blog: huggingface.co/blog/leaderboa…

NEW PAPER out today in @BMJ_latest TRIPOD+AI: reporting recommendations for studies developing or validating prediction models for use in healthcare that use #machinelearning methods

bmj.com/content/385/bm…

#ArtificialIntelligence #AIstandards #OpenAccess

Please share 🙏

Our new work in Nature Medicine

Using generative models in path, radiology and derm - we can create synthetic training data to ⬆️ AI fairness & robustness, including underrepresented groups. Work w/ a great team Google DeepMind Google AI Google Health

nature.com/articles/s4159…

This is gem. Immediately sold when seeing Rule No 1 'The application is for the reviewer, not you, the applicant—remember that'. Elements of successful NIH grant applications | PNAS pnas.org/doi/10.1073/pn…

Our new study estimates that ~17% of recent CS arXiv papers used #LLMs substantially in its writing. Around 8% for bioRxiv papers arxiv.org/abs/2404.01268 🧵

Are medical studies being written with ChatGPT?

Well, we all know ChatGPT overuses the word 'delve'.

Look below at how often the word 'delve' is used in papers on PubMed (2023 was the first full year of ChatGPT).

I study causal informatics. Period. Thanks Miguel Hernán for making my future self-introduction so much easier, and for a great keynote talk with successes and failures towards reliable RWE! #IS24 #AMIA

science is about seeing things

LLMs have made impressive progress on medical benchmarks like MedQA since 2021. Both open and closed medical LLMs have focused a lot on MedQA performance, but this has a number of problems when we think about deploying LLMs in hospitals. /1

When ppl ask me about my priorities as a Chief Health AI Officer, I share the Health AI Paradox.

Researched models aren't implemented. Implemented models aren't researched.

My priorities are simple. Help researched models get implemented, and implemented models get researched.

Hot off the press JAMA Internal Medicine

AEDs are only effective if put to use. Additional efforts are needed to overcome barriers for public access defibrillation.

Article:
jamanetwork.com/journals/jamai…

Editorial:
jamanetwork.com/journals/jamai…

U.S. News & World Report:
usnews.com/news/health-ne…

“invested over a million dollars in developing a finance-domain focused LLM named BloombergGPT

within just six months … a model (AdaptLLM-7B) costing merely $100 came out surpassing BloombergGPT in performance.”

Tech is brutal lol

PMC-Patients, an open dataset of 167k patient summaries and their relations, finally published at nature.com/articles/s4159….

Many exciting things to try!

⁉️Does GPT-4 perpetuate racial & gender biases in clinical applications?

We evaluated GPT-4 in 4 scenarios: med education, diagnostic reasoning, plan generation, & patient assessment. TLDR: GPT-4 exhibits subtle, but systemic signs of bias.
Led by Travis Zack & Eric Lehman
👇

😱 500k Echocardiograms to be released in #MIMIC in January!!

Large datasets of ECG linked w/ Echocardiograms at the patient level

👉 huge opportunity for #ML researchers to model heart function (eg EF) from cheaply-acquired ECG data