#cliss2024
#BigScience
#clearlakeca

#cliss2024
#bigscience
#ClearLakeCA

#cliss2024
#bigscience
#clearlakeca

How is the BigScience 176B model trained: a visual overview of the hardware and parallelism setup

It's that time of year! Learn all about Harmful Algal Blooms and who's tracking them in Clear Lake at the CLISS 2024! Until then, check out Clear Lake Water Quality page that posts all current HABS results.

#cliss2024
#BigScience
#ClearLakeCA

A few days left until the first BigScience follow-up project 🚀 Any guesses ?

DeepMind's Gopher and BigScience's BLOOM already use relative position embeddings, but most other language models don't. I believe we should all start using relative positioning.

In this new post, I discuss the use case for relative position methods:
ofir.io/The-Use-Case-f…

ローカルでLLMを動かすやつのBloomz版とも言えるbloomz.cppを初代M1 macbook air（メモリ16GB）で動かしてみました。動いた！ 試したのは70億パラメータの bigscience/bloomz-7b1 。アメリカの大統領を聞いたらG.W.ブッシュを答えたけど。レスポンスも7Bなら早いです
github.com/NouamaneTazi/b…

BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model
abs: arxiv.org/abs/2212.04960

Wondering how one can create a dataset of several TB of text data to train a language model?📚

With BigScience Research Workshop, we have been through this exercise and shared everything in our #NeurIPS2022 paper 'The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset'

🧵

With the ability to train self-supervised models, research is scaling up both data and model size at an impressive rate. Hugging Face BigScience is a new effort to establish good practices in data curation.
bit.ly/3xog6uR
#dcai #SelfSupervised #labeling #data #ai #ml

1700億パラメータのOSS LLM。性能も良さげ。300GBくらいあるっぽい。
bigscience/bloom · Hugging Face huggingface.co/bigscience/blo…

Excited to announce the BigScience Biomedical Hackathon! Together we're creating an open source, community resource of over 150 biomedical datasets. Join us! 🙌

🌸 Our mission: hfbigbio.github.io
🚀 Contribute: github.com/bigscience-wor…

Great presenting my paper on #conflicts in #BigScience Global Transformations and Governance Challenges conference. Thanks to my co-panelists Babak Rezaee, Dr Dominika Czerniawska & Inga Ulnicane for their useful feedback & the lively discussion.

The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset

abs: arxiv.org/abs/2303.03915

See the Yohe Lab featured here in contributing to the new UNC CHARLOTTE COLLEGE OF COMPUTING + INFORMATICS BioinformaticsUNCC UNCC Biological Sciences Center for Computational Intelligence to Predict Health and Environmental Risks ( #CIPHER )! 🦇👩‍🔬🦠🔬👩‍💻📊
features.charlotte.edu/laurel-yohe

The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset

Documents the data creation and curation efforts of ROOTS corpus, a 1.6TB dataset used to train BLOOM

Releases a large initial subset of the corpus

data: huggingface.co/bigscience-data
abs: arxiv.org/abs/2303.03915

Tule Talk: Learn about shorelines and the animals that love them at the CLISS 2024!

#cliss2024
#BigScience
#ClearLakeCA

🎊15 years ago today Target Station 2 detected it's first neutrons!! 🎉
To celebrate here is a collection of some of our favourite science highlights from each of our fantastic TS2 Instruments across the years... isis.stfc.ac.uk/Pages/15-Years…

#HappyBirthday #BigScience Science and Technology Facilities Council

Another LLM not instruction tuned on OpenAI-derived data, but on P3 huggingface.co/datasets/bigsc… (came from BigScience T0 arxiv.org/abs/2110.08207)

After Dolly v2: Pythia-12b instruction tuned on Databricks' in-house dataset (v1 was Alpaca, derived from OpenAI) databricks.com/blog/2023/04/1…