Skip to content

Commit

Permalink
Add MatX logo.
Browse files Browse the repository at this point in the history
  • Loading branch information
reinerp committed Jul 11, 2024
1 parent 871bd91 commit f89f6e4
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 0 deletions.
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
<p align="center">
<img src="docs/matx.svg" width="200"/>
</p>

# seqax = sequence modeling + JAX

seqax is a codebase for small-to-medium-scale LLM pretraining research. The entire training program---including the model implementation; optimizer; multihost FSDP and tensor parallel partitioning---is [500 lines of code](/train.py), which scales well up to ~100 GPUs or TPUs[^1] and [typically achieves good MFUs of 30-50%](#performance).
Expand Down
1 change: 1 addition & 0 deletions docs/matx.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit f89f6e4

Please sign in to comment.