Download
Open-source pre-training implementation of Google's LaMDA research paper in PyTorch. The totally not sentient AI. This repository will cover the 2B parameter implementation of the pre-training architecture as that is likely what most can afford to train. You can review Google's latest blog post from 2022 which details LaMDA here. You can also view their previous blog post from 2021 on the model.
Features
- Open-source pre-training implementation of Google's LaMDA research paper in PyTorch
- T5 Relative Positional Bias in Attention
- Gated GELU Activation in the Feed forward layer
- GPT-like Decoder Only architecture
- Autoregressive with Top-k sampling
- Sentencepiece Byte-pair encoded tokenizer
Project Samples
License
MIT LicenseFollow LaMDA-pytorch
Other Useful Business Software
Solve Your Real-world Business Challenges From A Single Timesheet With Powerful Built In Integration
TimeControl is a multi-purpose timesheet system designed to serve both Finance and Project Management. TimeControl has been designed to serve many purposes simultaneously. TimeControl tracks time on a task-by-task, project-by-project basis. Yet, despite its project-based controls, it remains a financial timesheet with all the controls necessary to fulfill the stringent needs of payroll, human resources, billing and finance. TimeControl is available both for subscription in the cloud or for purchase for an on premise installation and includes both a browser interface and the free TimeControl Mobile App for iOS and Android devices.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of LaMDA-pytorch!
