Communication is an interesting topic because of all the innovative solutions we design for it and their surprising results. Language, media, and the Internet are our attempts to communicate with each other, and everyone uses them in their unique way.
I'm keeping a list intersting works in communication here.
- Shannon's theory of information: Modeling communication mathematically helped build the foundations of telecommunication networks.
- Vannevar Bush's Memex Modeling information access and associative nature of information retrieval based on how our memory works. This was like a very basic implementation of the Internet and hyperlinks.
- Wolfram Language Wolfram language is a high-level programming language to model information and communication for symbolic computers. Some projects, like Wolfram, don't separate communication from computing.
- Dynamicland It's another project that combines communication with processing but also moving toward human computer interaction (HCI).
- ChatGPTUsing transformers' architecture and attention mechanism to develop general but meaningful connection among concepts. Transformers' architecture was introduced by Google and disrupted many fields in computing and AI. This product is interesting because it attempts to use language for modeling everything and eventually is becoming a computation platform, although it's very inefficient for computing even compared to most inefficient computing models.
- Hyena Hayena is another attempt at the attention mechanism but instead of common transformers' architecture it models attention "by interleaving implicitly parametrized long convolutions and data-controlled gating". This is important because first it still proves how effective is the attention mechanism in building general yet accurate relationship between concepts (which is key for communication) and second it's more accurate and more effchient than models like GPT family.