@joe@f.duriansoftware.com It’s funny how many of the tricks used to make language models work well on natural language reintroduce some of the limitations of humans, e.g. with subject-verb dependency distance, even though an attention mechanism with a gigantic context doesn’t have the same working memory limitations as humans.