LongNet: Scaling Transformers to 1,000,000,000 Tokens Code release: https://github.com/microsoft/torchscale July 2023: release preprint LongNet: Scaling Transformers to 1,000,000,000 Tokens
