Posts on In Search of Great Ideas
https://eitanporat.github.io/posts/
Recent content in Posts on In Search of Great IdeasIn Search of Great Ideashttps://eitanporat.github.io/%3Clink%20or%20path%20of%20image%20for%20opengraph,%20twitter-cards%3E
https://eitanporat.github.io/%3Clink%20or%20path%20of%20image%20for%20opengraph,%20twitter-cards%3E
Hugo -- gohugo.ioSun, 10 Mar 2024 00:00:00 +0000Learning is Compression
https://eitanporat.github.io/posts/learning-is-compression/
Sun, 10 Mar 2024 00:00:00 +0000https://eitanporat.github.io/posts/learning-is-compression/In this blog post, we explore how compressing data is equivalent to learning data distributions. Not in the sense that neural networks are compressing data in their weights, but in an information theoretic sense. Following up on Shannon's work, we use a GPT-2 model from Karpathy's nanoGPT repo to create a compression algorithm which combines Shannon encoding with next-token prediction. Using this model, we get a compression algorithm which beats the native python implementation (18.9% vs 44.73%)The Ergodic Theorem
https://eitanporat.github.io/posts/ergodic-theory/
Sat, 18 Mar 2023 00:00:00 +0000https://eitanporat.github.io/posts/ergodic-theory/What is Ergodic Theory? In this post, I present some wonderful and intuitive proofs for the ergodic theorems and describe two applications of the ergodic theorem.A Remarkable Connection Between Communication Complexity and Circuit Depth
https://eitanporat.github.io/posts/communication-complexity-and-monotone-circuits/
Sun, 26 Feb 2023 00:00:00 +0000https://eitanporat.github.io/posts/communication-complexity-and-monotone-circuits/Why is there a connection between two seemingly disparate concepts from complexity theory? What are the key insights leading to the proof?