About me

I am a deep learning enthusiast, striving to bring intelligent systems into game development. I enjoy creating games and training deep reinforcement learning agents to play them. I believe that games make great environments for reinforcement learning research because of the variety of possible tasks and the relative cheapness of development, training, and deployment.

My specialty is the design of reinforcement learning environments with game engines, primarily Unity. I’ve been using the ML-Agents toolkit since it was in early beta and have thoroughly studied the complexities of communicating data between game engines and Python, training parallelization, and the training algorithms themselves.

I experimented with applications of reinforcement learning to various games, and I learned that the design of the way that the agent interacts with the world is a crucial but often overlooked part of reinforcement learning. Researchers often work with pre-made environments, but any real task involves design of the environment itself. I make custom sensors, action and reward systems, and lay out the decision-making process to make agents that behave in the desired ways.

To lay the groundwork for my knowledge, I completed the Udacity Deep Reinforcement Learning Nanodegree and read books on deep learning, reinforcement learning, machine learning, and studied the required math. I spend all my spare time looking for interesting papers and projects to keep myself up-to-date with these constantly evolving fields. Over the years, I have gathered a large collection of noteworthy projects, which I am always happy to talk about and want to share with the community in the near future.

The topics that I am interested in right now are population based training with network distillation and efficient interaction with continuous time environments. I experiment with them in different projects that I’m either doing myself or volunteering for.

I am interested in bringing my experience outside of game engines too. Deep learning is an incredibly interesting topic on its own. Its mathematical background is fascinating, and Python, Tensorflow, and PyTorch are tools that I use for my work.

Background

I started learning about how video games work in high school. Back then, I used to host heavily modded servers for games like Team Fortress 2, Left 4 Dead 2, Killing Floor 2, and, of course, Minecraft. Digging in the files, figuring out how it all works and seeing a small community form around my servers brought me joy and encouraged me to keep learning.

Even though I was planning to pursue an education in computer science, due to circumstances, I ended up in the field of hospitality business and management. Working in operations and constantly dealing with people was new to me, and I believe that it taught me important social skills that serve me well to this day.

After I got my diploma in 2016, I started learning about video game development with Unreal Engine, at first as a hobby, but I quickly fell in love with it. I always liked science, and game development quickly showed me how much I missed working with software. I spent some time learning about different parts of game engines and learned some C++, but I have been blown away by the simplicity of the Blueprints and how much and how quickly can be achieved with them. I signed up for a week-long in-person course on Unreal Engine, where I met similar-minded people, did a few projects with them, and it became a point when I decided that this is what I would like to do professionally.

I noticed the field of deep learning around the same time, and it immediately caught my attention. The achievements of OpenAI and DeepMind were astounding, and I used my game development and programming skills to create my own small neuroevolution project. It was also the time when I first learned about connecting game engines to Python and wrote my first lines of Tensorflow code.

I enrolled in a Game Design and Development program at Centennial College in Toronto and moved there to make it a full-time job. The program covered all aspects of game development in Unity, and I put emphasis on learning C# and deep reinforcement learning. During my studies, I took part in various game jams and projects, which taught me the practical aspects of game development.

Since I’ve graduated I have been focusing on learning Python and deep reinforcement learning related topics. I also made a couple projects where I tested different hypothesis related to the environment design and volunteer to develop a learning based AI for an RTS game “Sanctuary”. Writing articles is also one of the directions that I am currently exploring.

There were 42 “I” and 25 “learn” throughout this text, I hope it didn’t annoy you too much :)