Segment Trees

Segment tree is used for storing information about intervals. Its very useful when doing range queries.

[Read More]

Python Concepts

My attempt to explain some python concept after learning them. Currently, the following concepts are explained:

  • Python dunder methods
  • Class variables
[Read More]

Cpp concepts

My attempt to explain some c++ concept after learning them. Currently, the following concepts are explained:

  • explicit
  • static
[Read More]

Attention

Attention mechanism [1,2] improved NLP architectures by allowing them to focus on a relevant part of input/representation similar to how we humans do. While reading a text if the first and last character of a word is correct, humans can understand the text [3]. This post examines the inner working of additive and multiplicative attention, i.e. How attention mechanism converts query and hidden states into attention scores.

[Read More]

Gradient estimation for hard non-linearities

The hard non-linearities such as step functions are useful for modelling a categorical variable in neural networks. The gradient is zero everywhere for hard activations, which leads to difficulty in training using backpropagation. We review some technique proposed to estimate gradient such as straight through estimator.

[Read More]

Memory networks

Memory Networks are neural networks which have access to explicit storage and a mechanism to access storage to answer query requiring multi-hop reasoning.

/Big_Oak.png
[Read More]

WIP:A short tour of libpst

The libpst is a library to extract text from microsoft properitery format .pst, which is used to storing emails, attachments. There isn’t a good documentation for libpst. This is a simple tour writing a c++ program from scratch to extract an email and it’s attachment from .pst file using libpst.

[Read More]