There's more detailed examination of Transformers from circuit complexity in thi... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		shawntan on April 18, 2023 \| parent \| context \| favorite \| on: The New XOR Problem There's more detailed examination of Transformers from circuit complexity in this paper: https://arxiv.org/abs/2106.16213 There are a few references also in the blog post.

krackers on April 18, 2023 [–]

Thank you! I'd love more articles analyzing LLMs from a circuit-complexity perspective!

Also seems modeling the transformer as in the TC0 class, in principle it should be able to do division and multiplication of numbers.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact