Gaurav Arora

Log of things I played with.

Segment tree is used for storing information about intervals. Its very useful when doing range queries.

My attempt to explain some python concept after learning them. Currently, the following concepts are explained:

• Python dunder methods
• Class variables

My attempt to explain some c++ concept after learning them. Currently, the following concepts are explained:

• explicit
• static

Attention mechanism [1,2] improved NLP architectures by allowing them to focus on a relevant part of input/representation similar to how we humans do. While reading a text if the first and last character of a word is correct, humans can understand the text [3]. This post examines the inner working of additive and multiplicative attention, i.e. How attention mechanism converts query and hidden states into attention scores.

The hard non-linearities such as step functions are useful for modelling a categorical variable in neural networks. The gradient is zero everywhere for hard activations, which leads to difficulty in training using backpropagation. We review some technique proposed to estimate gradient such as straight through estimator.

Memory Networks are neural networks which have access to explicit storage and a mechanism to access storage to answer query requiring multi-hop reasoning.

Below is a running list of interesting articles/StackOverflow answers which seems worth sharing and referring back.

The libpst is a library to extract text from microsoft properitery format .pst, which is used to storing emails, attachments. There isn’t a good documentation for libpst. This is a simple tour writing a c++ program from scratch to extract an email and it’s attachment from .pst file using libpst.

Can a single for loop be running with quadratic time complexity?