Anirudh's Blog on Anirudh Vemula

Revisiting Racket: Advent of Code 2025 Day 2

Tue, 02 Dec 2025 11:03:39 -0400

Day 2 of doing advent of code using Racket. Let’s go!

Before we discuss my solution, read the problem statement here. Now, for the first puzzle.

Puzzle 1

Parsing

As usual, we need to first start with parsing the input. The input is given as a single line with comma-separated ranges, where each range is of the form "start-end" where both start and end are numbers.

(define (strip string)
  (string-replace string "\n" ""))

(define (parse-range string-range)
  (let([range (string-split string-range "-")])
    (map string->number range)))

(define input (map (compose1 parse-range strip) (string-split (file->string "input.txt") ",")))

Few things to note here:

Revisiting Racket: Advent of Code 2025 Day 1

Mon, 01 Dec 2025 11:53:39 -0400

I’ve been meaning to revisit Racket for a very long time. It was the functional programming language that was taught in IIT Bombay Programming Paradigms course, which was my favorite computer science course of all time.

When I heard that Advent of Code was starting soon, I thought this was the best time to revisit Racket, and re-learn the language while solving all the puzzles.

Day 1 puzzles (two of them) were straightforward and can be found here. Read the problem statement before proceeding further as I discuss my solution in Racket below.

A Gentle Primer for Nonparametric Density Estimation: Kernel Density Estimation

Sat, 13 Apr 2024 16:53:39 -0400

In my previous post, we discussed a neat intuition for nonparametric density estimation and introduced a nonparametric method, histograms. This post describes another (very popular) density estimation method called Kernel Density Estimation (KDE).

Revisit Intuition

Let’s revisit the intuition that we developed for nonparametric density estimation. Given a region $R \subset \mathbb{R}^D$ of volume $V$, and that contains $K$ points from a sampled dataset of size $N$, we can estimate $p(x)$ for any point $x \in R$ as

A Gentle Primer for Nonparametric Density Estimation: Histograms

Sun, 07 Apr 2024 17:33:51 -0400

Parametric density estimation, a.k.a fitting probability distribution to observed data by tweaking the parameters of the distribution’s functional form, is all the rage now with generative modeling and LLMs. I think nonparametric methods deserve some love too and I hope to give a very small primer on these methods in a series of posts.

What are Nonparametric methods?

Looking at the name, you might have guessed that these are methods where you fit a probability distribution to data without any underlying parameters that are computed from the data itself.

Lessons from AlphaZero: Part 3

Wed, 13 Dec 2023 23:47:09 -0400

We will continue with part 3 (and the final part) of the blog post series on this book by Bertsekas. As always, I would highly recommend the readers to start with part 1 and part 2 before continuing in part 3, as all of the notation used in this post is introduced in the previous parts.

As a recap, here’s the breakdown for the 3-part blog series:

Part 1: Notation, Bellman Operators, and geometric interpretation of value iteration.
Part 2: Approximation in Value Space, one-step and multi-step lookahead, and certainty equivalent approximations.
Part 3 (this post): Geometric interpretation of policy iteration, truncated rollout, and final remarks.

Let’s begin!

Lessons from AlphaZero: Part 2

Sat, 02 Dec 2023 23:47:09 -0400

This is part 2 of the blog post series on this book from Bertsekas, and its key takeaways.

Let’s pick it up from where we left off in part 1. Will highly recommend readers to check out part 1 before continuing in part 2, as some important notation and ideas were described there, which will be relevant in this part.

As a recap, here’s the breakdown for the 3-part blog series:

Part 1: Notation, Bellman Operators, and geometric interpretation of value iteration.
Part 2 (this post): Approximation in Value Space, one-step and multi-step lookahead, and certainty equivalent approximations.
Part 3: Geometric interpretation of policy iteration, truncated rollout, and final remarks.

With that, let’s kick-off part 2.

Lessons from AlphaZero: Part 1

Fri, 24 Nov 2023 23:47:09 -0400

I recently came across this book from Bertsekas that looked very interesting. It promises to provide a novel (geometric) interpretation of the methods that underlie the success of AlphaZero, AlphaGo, and TD-Gammon. I thought it would be interesting to take this as an opportunity to read the book, and have a blog post series on the key takeaways.

I think this will be a 3-part blog series with the following contents:

AdaGrad

Wed, 18 Nov 2020 23:47:09 -0400

Francesco Orabona’s monograph on online learning has a very neat and intuitive way of looking at AdaGrad. I found it to be really useful.

Looking back at OGD

From our note on online (sub)gradient descent, we had the following regret bound

$$R_T(u) \leq \frac{D^2}{2\eta_T} + \sum_{t=1}^T \frac{\eta_t}{2} ||g_t||_2^2 $$

where $D$ is the diameter of the closed, convex set $V$. If we assume a fixed learning rate for all time steps and optimizing we get the optimal learning rate as

CMAX++: Leveraging Experience for Planning and Execution with Inaccurate Models

Tue, 13 Oct 2020 14:39:32 -0400

In this blog post, I will summarize the key contributions of my recent submission on CMAX++. This work mainly improves upon my previous work, CMAX (highly recommended to read a summary of CMAX in this blog post.)

The broad theme of both these approaches is to use simplified and inaccurate dynamical models to significantly reduce the amount of real world experience needed by the robot to complete a task. The setting we consider is a goal-oriented task setting where the robot needs to reach the goal online, without any resets. This setting is classically considered using the planning and execution framework where the robot constantly replans after each execution to intelligently use any experience obtained in the past executions. It goes without saying that this setting is extremely challenging as the initial model used for planning cannot capture the real world dynamics accurately and the robot does not have access to any resets to “undo” any past mistakes.

Regret Analysis of Stochastic Bandit Problems

Thu, 23 Jul 2020 19:09:28 -0400

$K$ arms, $T$ rounds (assume known $K$ and $T$)
Goal is to maximize total reward over $T$ rounds

Assumptions

Bandit Feedback: only observe reward for the arm played
IID rewards: For each arm $a$, $\exists$ a reward distribution $D_a$ which is unknown
Per-round rewards are bounded, i.e. $r_t \in [0, 1]$ for $t \in [T]$

We are primarily interested in mean vector $\mu \in [0, 1]^K$ where $\mu(a) = \mathbb{E}[D_a]$

Notation

Set of arms $\mathcal{A}$
Best mean reward $\mu^* = \max_{a \in \mathcal{A}} \mu(a)$
Gap of arm $a$: $\Delta(a) = \mu^* - \mu(a)$
Optimal arm $a^* $, i.e. $\mu(a^*) = \mu^*$

Regret

$$ R(T) = \mu^*T - \sum_{t=1}^T \mu(a_t) $$

Online Learning for Adversaries with Past Memory

Thu, 09 Jul 2020 22:33:34 -0400

The framework of OCO with memory is:

At each round $t$, player chooses $x_t \in K \subset \mathbb{R}^n$
Then, a loss $f_t: K^{m+1} \rightarrow \mathbb{R}$ is revealed and player suffers loss of $f_t(x_{t-m}, \cdots, x_t)$
Assume $0 \in K$, and $f_t(x_0, \cdots, x_m) \in [0,1]$ for any $x_0, \cdots, x_m \in K$
Assume that after $f_t$ is revealed, the player is aware of the loss she would suffer had she played any sequence of decisions $x_{t-m}, \cdots, x_t$ (counterfactual feedback model)

Minimize policy regret $$R_{T, m} = \sum_{t=m}^T f_t(x_{t-m}, \cdots, x_t) - \min_{x \in K} \sum_{t=m}^T f_t(x, \cdots, x) $$

Online Newton Step

Sat, 04 Jul 2020 11:12:37 -0400

Exp-concave Functions

OGD achieves logarithmic regret for strongly convex and lipschitz functions (see this note), but this is a rather strong condition.

For twice-differentiable strongly convex functions, we require that the hessian is positive definite and has full rank!

However, sometimes the hessian might not be full rank (sometimes, even rank one) but is large in the direction of the gradient. This is a property of exp-concave functions.

Exp-concave Definition: A convex function $f:\mathbb{R}^n \rightarrow \mathbb{R}$ is $\alpha$-exp-concave over $K \subseteq \mathbb{R}^n$ if the function $g$ is concave, where $g:K \rightarrow \mathbb{R}$ is $$ g(x) = \exp(-\alpha f(x)) $$

Online Gradient Descent with Strongly Convex Functions

Fri, 03 Jul 2020 16:52:01 -0400

Turns out we can do better $\sqrt{T}$ regret we saw for OGD analysis for convex lipschitz losses if they possess some curvature that can be exploited.

Strong Convexity

$f$ is $\mu$-strongly convex over convex set $V$ w.r.t norm $||\cdot||$ if $\forall x, y \in V$ and $g \in \partial f(x)$ $$ f(y) \geq f(x) + \langle g, y - x\rangle +\frac{\mu}{2}||x - y||^2$$

In other words, a strongly convex function is lower bounded by a quadratic (instead of linear like convex functions). Hence, we have a tighter lower bound. Actually, several possible quadratic lower bounds since there can be more than one subgradient.

Online Gradient Descent

Wed, 01 Jul 2020 12:57:48 -0400

Similar to SGD but different: loss functions are different at each time step

Algorithm

Projected Online Gradient Descent: $x_1 \in V \subseteq \mathbb{R}^d$ where $V$ is a closed, convex set and $\eta_1, \cdots, \eta_T > 0$

for $t = 1, \cdots, T$
Play $x_t$
Receive $\ell_t: \mathbb{R}^d \rightarrow (-\infty, \infty]$ and pay $\ell_t(x_t)$
Set $g_t = \nabla \ell_t(x_t)$
$x_{t+1} = \Pi_V(x_t - \eta_t g_t) = \arg\min_{y \in V}||x_t - \eta_tg_t - y||_2$
end

Regret Analysis

Crucial to the analysis is the following simple lemma:

Frank-Wolfe a.k.a Conditional Gradient Descent

Sat, 27 Jun 2020 22:15:13 -0400

An alternative approach to projected gradient descent for minimizing a smooth convex function $f$ over a compact, convex set $K$.

At the $t$-th step, it considers the first-order taylor approximation $\hat{f}$ of $f$ around $x_t$, and minimizes $\hat{f}$ over $K$, obtaining a minimizer $y_t$. $$ \hat{f}(y) = f(x_t) + \langle \nabla f(x_t), y - x_t \rangle $$ $$ y_t = \arg\min_{y \in K} \hat{f}(y) = \arg\min_{y \in K} \langle \nabla f(x_t), y \rangle $$

Forward and Reverse KL Divergence

Fri, 26 Jun 2020 23:47:09 -0400

KL-divergence is given by: $$ D_{KL}(P||Q) = \mathbb{E}_{x \sim P}\left[\log \frac{P(X)}{Q(X)}\right] $$

It is not necessarily symmetric, so $D_{KL}(P||Q) \neq D_{KL}(Q||P)$, thus it cannot be used as a distance metric.

It is always positive, i.e. $D_{KL} \geq 0$. Also, if there is a point $x$ where $Q(X) = 0$ but $P(X) \neq 0$, then $D_{KL}(P||Q) = \infty$. Thus, we need the support of $P$ to lie completely in $Q$ for a finite KL-divergence.

Hunter Thompson's letter on finding purpose

Wed, 17 Jun 2020 14:39:32 -0400

Several years ago, I came across this letter written by Hunter S. Thompson to his friend Hume Logan on finding purpose and meaning in one’s life. It had a profound impact on me back then, and very recently I read the letter again only to interpret it differently. The advice in the letter is timeless and I like to think that reading it again in a few years will reveal other hidden gems in this letter.

CMAX: Planning and Execution with Inaccurate Models

Wed, 06 May 2020 14:39:32 -0400

Very recently, I submitted a paper to RSS (update: accepted!) on how to use inaccurate models for planning in robotic tasks. For many robotic tasks, we usually have access to a (inaccurate) model of the robot and the environment. For example, simulators that employ sophisticated physics engines such as MuJoCo or Bullet can be used as forward dynamical models for planning. These models capture some complexities of the real world well, while they fail to capture other subtleties. However, they are still very useful sources of information on real-world dynamics that can be exploited to plan a good policy.

AISTATS 2019

Fri, 19 Apr 2019 15:59:13 -0400

Thoughts on the conference

My overall impression of AISTATS was that its a really good conference if statistical ML is what you are interested/working in. I learnt a fair bit about the basics of unknown topics and made a few friends who work in widely different areas. Its a single track conference so there’s more interaction among people and more exposure to the oral sessions. There’s barely any presence of industry (either in sponsors or papers) except for Google Research/Brain/Deepmind papers. And thankfully, AISTATS community seems to not be onboard the hype train (yet) and there’s ample time to ask questions in the poster session.

New York City in 35mm

Mon, 02 Oct 2017 15:59:13 -0400

Some of these photos were taken in New York city when I visited it in October for an AI forum. I wanted to take pictures in NYC for so long and felt this was as good a time as any. So, carried my old-ass camera around uptown and midtown manhattan, and took most of the photos.

Graffiti in Pittsburgh through 35mm

Fri, 22 Sep 2017 15:59:13 -0400

Early fall 2017, I went on a search for graffiti in Pittsburgh and found a lot! Took a bunch of snaps and developed them on Ilford HP5 35mm. To get most of these I had to bike around Pittsburgh (mostly in Lawrenceville and Oakland). Never knew Pittsburgh had so much street art.