Tech with Aziz

REALM : Retrieval Augmented Language Model Pretraining This paper was created by Google Research, and submitted in February 2020 and offers a new and enhanced way of language model pretraining. Also achieves SOTA on QA. Problem  : The paper starts by citing the issue with current language models and their pretraining. Bert, Roberta and T5 for example capture a great amount of world knowledge for a variety of NLP tasks however this knowledge is stored in model weights which makes it hard to interpret the model’s results and the model is not modular. To capture more knowledge one has to simply increase the numbers of parameters, data and train for longer steps which can be very costly. Idea  : To fix the issues above the paper presents a new way of pretraining language models to be as performing or better on NLP tasks with fewer parameters. The idea is fairly simple, let’s say you have a question and you want to answer it, the first thing that a human would do is to check g...

  Arrays in Python Explained!  If you are getting into programming or learning about Python, this article is for you, arrays are one of the most used data structures. How does Python represent arrays ? How can we manipulate arrays in Python to solve different issues ? This is what we are going to find out in this article.  How Do Computers Store Information? As you know the most primary form of information in a computer is composed of  bits  of information. 8 bits of these information are then grouped to form what we call a  byte .  Because computers can store very large chunks of these bytes, they need a new mechanism that enables them to quickly read and write bytes, this is where  memory addresses  come into play, just like your personal home address differentiates where you live from your neighbor, the computer uses memory addresses to differentiate where different pieces of data and information live. It’s also worth mentioning ...